At its simplest having bitcoin means that at some address a balance is recorded in the blockchain (often comprised of several smaller input balances) and you have control over the private key necessary to sign a transaction which can assign the balance at this address to one or more other addresses. Everything construction beyond this that contributes to the idea of having a bitcoin wallet is an abstraction, typically the end goal of these constructions is making this bitcoin deal a bit easier to understand and use.
The simplest wallet construction and the one implemented bitcoin clients like Bitcoin-Qt/bitcoind, MultiBit, Armory, and Electrum stores your keys and addresses, somehow gathers information from the blockchain on the balances assigned to you addresses so that it can present you number of bitcoins you have, and it can broadcast a transaction to the network including input balances from one, some, or all of your addresses in the "wallet" it manages for you. You can back up your keys elsewhere on storage mediums and methods ranging from saved on USB thumb drives to paper hand written on a napkin to laser etched on a tablet of saphire, but having software that can manage and use these private keys is awfully, useful? A safe full of saphires bearing private keys might constitute a form of bitcoin wallet as well since it collects information necessary to spend balances attached to bitcoin addresses, but generally the idea of complete wallet is going to include some provision for monitoring and spending balances. Paranoid/Prudent security precautions might mean spreading one's wallet functions between online and offline things.
Now other things that might hold Bitcoin balances for a person and let them send bitcoins to an address, but they control the private keys. Some of these things are Sports Books, Some of these things are exchanges, and still other things are off-chain payment networks (this last one is what I'd consider a service like Inputs.io). Generally they need the private keys to do whatever it is they do, but it is best not to think of these things as your wallet. If you are new to bitcoin, plan to read more, and aren't comfortable keeping your own private keys yet maybe use of of these things that advertises itself as a wallet as one until you can trust to keep your private keys safe. If you are the sort who insists on keeping an NSA office in your pocket consider one of these things as a potential way to keep a small sum of bitcoin you can spend on your mobile device. Understand though that as understood by the bitcoin network balances held by these services are ultimately their balances, it is simply the case that reliable operators will give you the courtesy of spending balances they credit to you out of their services.
Note: This post can probably stand on its own as a statement on the idea of a bitcoin wallet as I view it, but I wrote it to follow Anti-Bitcoin “Bitcoin Apps” and will follow it with a further post or two on how Gliph needs to suck dicks (though smartphone apps can not indeed suck dicks, despite VC wet dreams to the contrary).