I found speed and survival good enough to not neglect. They are not as important as strength, agility and constitution, but also not useless for a warrior (like intelligence is).
Survival's primary effect is on initiative, which determines who goes first during battle and influences each ‘round’ of the fight. Speed is the major influence on evasion, and the minor contributor to initiative. Evading hits is of course preferred, though better armor and more healing potions can help compensate for being hit more often. Your attack rate and recovery time are both influenced by speed.
If you are not frequently getting hit first and/or consistently, or taking repeated blows before being able to counter-attack, then you can probably keep ignoring speed and survival.
BD's stat system allows much more flexibility in character development than DD, and this results in many more viable builds (in DD almost every warrior ended up with a distribution of 2 parts strength, 2 agility and 1 constitution). If what you are doing keeps working for you, then keep doing it.
