Have you considered using SQL Server Compact with database encryption?
SQL Server Compact is a file-based database, not server-based, which as you said in a comment is preferable for you.
The file on disk is encrypted. It is decrypted in memory, decrypted data is not stored to disk.
It's extremely simple to work with - you just need to put the password in the connection string when you open a connection to the database.
Seems this would fit your use case.
An aside on Password/Key management: Yeah, thanks Alexei, I was hoping nobody would mention that one. :-) It's a tough problem, especially if we're talking about an application that is distributed to the people who you want to protect the database from.
Ultimately, it can't be done. If your application can open the database, and you distribute your application, an attacker can reverse-engineer your application and figure out how to open the database. .NET apps are of course particularly easy to decompile.
The way I see it, you can either just put the key in the code, and accept that anyone who decompiles the code will find it; or you can try to make efforts to obfuscate it (break up the key into various places in the code, write ugly complicated code to reconstruct it, etc.) If you do that, it will still be breakable - you'll just raise the bar from "anyone who decompiles the code" to "anyone who decompiles the code and is willing to spend the time & effort working through your obfuscation".