Before owning any bitcoin, you need somewhere to store them. That place is called a "wallet." Rather than actually holding your bitcoin, it holds the private key that allows you to access your bitcoin address (which is also your public key). If the wallet software is well designed, it will look as if your bitcoins are actually there, which makes using bitcoin more convenient and intuitive.
Actually, a wallet usually holds several private keys, and many bitcoin investors have several wallets.
Wallets can either live on your computer and/or mobile device, on a physical storage gadget, or even on a piece of paper. Here we'll briefly look at the different types.
Electronic wallets
Electronic wallets can be downloaded software, or hosted in the cloud. The former is simply a formatted file that lives on your computer or device, that facilitates transactions. Hosted (cloud-based) wallets tend to have a more user-friendly interface, but you will be trusting a third party with your private keys.
Software wallet
Installing a wallet directly on your computer gives you the security that you control your keys. Most have relatively easy configuration, and are free. The disadvantage is that they do require more maintenance in the form of backups. If your computer gets stolen or corrupted and your private keys are not also stored elsewhere, you lose your bitcoin.
They also require greater security precautions. If your computer is hacked and the thief gets a hold of your wallet or your private keys, he also gets hold of your bitcoin.
The original software wallet is the Bitcoin Core protocol, the program that runs the bitcoin network. You can download this here (it doesn't mean that you have to become a fully operational node), but you'd also have to download the ledger of all transactions since the dawn of bitcoin time (2009). As you can guess, this takes up a lot of memory - at time of writing, over 145GB.
Most wallets in use today are "light" wallets, or SPV (Simplified Payment Verification) wallets, which do not download the entire ledger but sync to the real thing. Electrum is a well-known SPV desktop bitcoin wallet that also offers "cold storage" (a totally offline option for additional security). Exodus can track multiple assets with a sophisticated user interface. Some (such as Jaxx) can hold a wide range of digital assets, and some (such as Copay) offer the possibility of shared accounts.
Online wallet
Online (or cloud-based) wallets offer increased convenience - you can generally access your bitcoin from any device if you have the right passwords. All are easy to set up, come with desktop and mobile apps which make it easy to spend and receive bitcoin, and most are free.
The disadvantage is the lower security. With your private keys stored in the cloud, you have to trust the host's security measures, and that it won't disappear with your money, or close down and deny you access.
Some leading online wallets are attached to exchanges (such as Coinbase and Blockchain). Some offer additional security features such as offline storage (Coinbase and Xapo).
Mobile wallets
Mobile wallets are available as apps for your smartphone, especially useful if you want to pay for something in bitcoin in a shop, or if you want to buy, sell or send while on the move. All of the online wallets and most of the desktop ones mentioned above have mobile versions, while others - such as Abra, Airbitz and Bread - were created with mobile in mind.
Hardware wallets
Hardware wallets are small devices that occasionally connect to the web to enact bitcoin transactions. They are extremely secure, as they are generally offline and therefore not hackable. They can be stolen or lost, however, along with the bitcoins that belong to the stored private keys. Some large investors keep their hardware wallets in secure locations such as bank vaults. Trezor, Keepkey and Ledger and Case are notable examples.
Paper wallets
Perhaps the simplest of all the wallets, these are pieces of paper on which the private and public keys of a bitcoin address are printed. Ideal for the long-term storage of bitcoin (away from fire and water, obviously), or for the giving of bitcoin as a gift, these wallets are more secure in that they're not connected to a network. They are, however, easier to lose.
With services such as WalletGenerator and BitcoinPaperWallet, you can easily create a new address and print the wallet on your printer. Fold, seal and you're set. Send some bitcoin to that address, and then store it safely or give it away. (See our tutorial on paper wallets here.)
Are bitcoin wallets safe?
That depends on the version and format you have chosen, and how you use them.
The safest option is a hardware wallet which you keep offline, in a secure place. That way there is no risk that your account can be hacked, your keys stolen and your bitcoin whisked away. But, if you lose the wallet, your bitcoin are gone, unless you have created a clone and/or kept reliable backups of the keys.
The least secure option is an online wallet, since the keys are held by a third party. It also happens to be the easiest to set up and use, presenting you with an all-too-familiar choice: convenience vs safety.
Many serious bitcoin investors use a hybrid approach: they hold a core, long-term amount of bitcoin offline, while having a "spending balance" for liquidity in a mobile account. Your choice will depend on your bitcoin strategy, and your willingness to get "technical."
Whatever option you go for, please be careful. Back up everything, and only tell your nearest and dearest where your backups are stored.
For more information on how to buy bitcoin, see here. And for some examples of what you can spend it on, see here.
(Note: specific businesses mentioned here are not the only options available, and should not be taken as a recommendation.)