Why does checking or unchecking "Show EditMe Menu By Default" have no effect?

In the Site Settings screen, the Show EditMe Menu By Default" checkbox refers to the tools menu that shows on every page of your site, and contains the Edit Page, Login and other links. Checking this box indicates that EditMe should show the Tools menu by default when a new visitor views the site.

EditMe stores the current show/hide state of this menu in a cookie. As the administrator, you have presumably visited the site and shown or hidden the menu. When you did, EditMe placed a cookie in your browser to remember your preference. When you view your site, you'll see your preference, not the default show/hide setting you've specified in Site Settings.

If you want to test the setting, you can delete your browser cookies and then visit the site, or visit the site with a browser that has never visited the site.