In my experience, prefixing x11vnc with sudo is required. Without it, I get an error similar to:
x11vnc -display :0 -auth /var/run/lxdm/lxdm-\:0.auth *** XOpenDisplay failed (:0)
x11vnc -display :0
However, the wiki says that using an X authority file requires root generally.
You may secure the server without using an X authority file, by creating a password file:
Then running x11vnc this way:
x11vnc -display :0 -rfbauth ~/.x11vnc/passwd