Vim: Can only undo most recent change

First thing, are you sure you are in vim-mode and not in vi-mode?

cat ~/.vimrc
set nocompatible