Some years ago favicons were only displayed in the address bar and bookmarks of your browser, but it has been showing up in more places, especially with the advent of start pages.
Here is a sample list of places where you will find favicons displayed next to your URL or page title (assuming your website has a favicon):
Next to your website’s URL in the address bar
This goes for Internet Explorer (IE 6,7), Firefox, Opera, and most other browsers
In your bookmarks (you usually have to visit the site from clicking the bookmark link once for it to show up properly)
Tabs when you’re “tabbed browsing”
This is by no means an exhaustive list, but it should give you an idea how important it is to have a favicon for your website these days. It’s an integral part of branding for your site and a pretty effective visual cue in your browser and numerous other places online.