I have a love/hate relationship with the ASP.NET Session. It's such a convenient place to put things, but when you start putting applications into production there are a number of less-than-obvious edge cases that can come up and bite you.
Most often the Session is used when managing state over a long process like a multi-step wizard or questionnaire. However, when people use the Session, they often lean on it a little. They'll bake it into their design so deep that when it doesn't work, they're screwed. That's not to say they shouldn't be able to lean on it, I'm just saying that there's a lot of things going on with Session (not just on ASP.NET, but other frameworks as well) in order to get it to look seamless.
BUILT IN OPTIONS
ASP.NET offers three options (four if you count rolling your own).
§ Inproc - The default, and usually works fine. However, you can get into trouble in a few scenarios.
§ Web Gardening - If you've setup IIS to run multiple instances of the IIS Worker Process on a single multi-proc machine, this is the equivalent of running a Web Farm, just on one machine. This technique is usually only useful when you've got a very CPU-intensive application - in other words, don't just turn on Web Gardening and expect your problems to get better instantly. It's subtle.
§ Unexpected Process Recycling - IIS6 had some wonky defaults and would recycle the AppPool or Process when some certain limits were hit, like after x number of requests or after 20 minutes. This is the classic "flaky session state is expiring" issue that lots of folks hit. You'll be more likely to see this if you've got really long running processes where users are logged in for long periods of time.
§ Out of proc - A good next step, this moves session out to a Windows Service. You can run one per Web Farm (meaning, you've got multiple machines but one instance of this service) and your session data will survive process recycles, but not system reboots. This is useful for both Web-Gardening and Web-Farming.
§ Folks usually forget to mark their objects as [Serializable] which basically gives your objects "permission" to leave their process space and be stored in memory in the State Service. If you've got a high-traffic site you might want to avoid storing complex objects and object graphs as you'll pay for it on the serialization. Of course, with all things, measure everything! You'll get best performance if you stick with basic types like strings, ints, etc.
§ UPDATE: I wanted to update this post and point folks to Maarten Balliauw's most excellent series on Out of Proc Session State (StateServer). He covers the basic setup, which is unremarkable, but then digs into the advanced stuff including "partitionResolvers" which I am ashamed to say I hadn't heard of! Recommend.
§ ASP.NET load balancing and ASP.NET state server (aspnet_state)
§ ASP.NET Session State Partitioning using State Server Load Balancing
§ SQL Server - The most robust, but now you'll pay for not only serialization, but storage. However, SQL Server is a highly tuned system and if you've got a site with any significant traffic I really recommend just skipping out-of-proc and putting your session state into a SQL Server with a lot of memory. Rather than trusting ASP.NET out of proc Session State Server to be a small database, leave the database work to the databases.
§ The benefits of SQL Server for your Session State include surviving process recycles and reboots. but more importantly using removes a lot of variables from your troubleshooting in the sense that you no longer worry about the storage of your Session, now you just need to worry if your Session Cookies are getting passed back and forth from browser to server.
§ Make sure you're using Windows Integrated Security and that you decide if you want ASP.NET to store Session in tempdb (which won't survive a SQL recycle) or a dedicated database (my recommendation).
There's a number of things that can go wrong, some of which I mention above, but here's what I usually run through when troubleshooting things.
§ Is the ASP.NET SessionID Cookie actually moving back and forth between browser and server. This can be confirmed by:
§ Using an HTTP Sniffer like ieHttpHeaders or HttpWatch or Fiddler and confirming that the Session ID cookie's value isn't changing between requests.
§ Confirming that the cookie isn't being blocked by IE, privacy settings, lack of a P3P policy document, local firewall like ZoneAlarm or Symantec, or a corporate proxy with an attitude problem.
§ Is IIS recycling the AppPool or Worker Process? Confirm the settings in IIS manager and make sure they are right for what you're doing.
§ Is the session timing out? Are you sure you're hitting the same VDir from whence you came and successfully resetting the sliding expiration on the Session ID?
§ Is some other thing like an Ajax call or IE's Content Advisor simultaneously hitting the default page or login page and causing a race condition that calls Session.Abandon? (It's happened before!)
At my last company Session became such a hassle for large high traffic applications that we just stopped using in-proc and started exploring alternatives.
SOME 3RD PARTY SESSION STATE OPTIONS
§ NCache from Alachisoft - An in-memory object cache that's distributed across your web farm. Think of it like Out of Process Session State, but distributed/clustered in their Enterprise Edition.
§ ScaleOut Software SessionServer - Fast, scalable in-memory storage that is distributed across machines. Full Disclosure: we worked with these guys while I was at Corillian, but never put them into production.
§ Memcached Session State Provider - Fahad has created ASP.NET Session State providers that will talk to memcached, a very popular distributed memory caching system originally created for LiveJournal.com and now used all over.