Yes, you can drop or move any items and they will remain where placed. While some people spread things out on the ground (gems can be used for lighting around your home base at night, for example), many prefer to collect a few chests (or other types of containers) to store stuff in.
You can set up a base anywhere you wish, though for convenience sake you should consider the proximity of the nearest teleporter pad and merchants. Some people consider aesthetics, as well (with a few using the cursed abbey for its atmosphere and relatively handy teleporter pad). I gave a screenshot and description of my base in the topic
Houses And their owners (playing again I'd probably move near Ars Magicana).
You can combine 2 stacks of straw to make a hay bed (if you combine them on the ground you can not move it afterwards, but it can be made inside a container or directly in your inventory). I think you have to drag and drop a bundle of straw onto an unbundled pile of straw, or something like that (click on a bundle of straw to unbundle it, and vise versa). There is also one movable bed in the game:
in a house a bit north of the cursed abbey.
As you progress in the game you'll likely build up enough restoration potions that you will no longer bother with sleeping.
You can buy a house in Verdistis after you do an associated quest. However, it is not in the greatest location, so I only bought it to complete that quest. Be careful not to move in early, though, since buying the house resets its status and at least in the disk version would remove items left in the chest there.
Most of the actual quest breakers mentioned in that topic were fixed. I think talking to Janus early after sneaking into the castle and accepting a quest is the only serious bug left (other problems exist, of course, but most are rare, minor and/or have fixes).
There is a sticky topic linking to a bunch of helpful topics, this one included.
