update: 20100503: It's been brought to my attention that there might have been a regression in one of the libraries, mesa, maybe, so that master might not be as reliable a reference point for this guide. so, just in case, here are the commit hashes that were used from the different projects:
- dri2proto: c34ce137fdb21fc9a52bb8d5a0c25e3c5d79e687
- drm: 6293152eb065016a2e5e4fcd047c2db5c2fb0f36
- linux-2.6 (nouveau master) f108a544dbc9b64b0b42cfc6e21bc4f5ffd38998
- mesa: dd3b98bdf67df9a5b1127bf7d50bbb6691597bfb
- xf86-video-nouveau: 13c10430ba8f7b0edff3ad8aae4a97672eea4a8f
for the non-git-savvy: git checkout -b mybranch <hash>
we now return to your regular broadcasting...
A week or two ago I figured it was about time that I took the plung and tried out the latest nouveau drivers from experimental on my laptop. Unfortunately, it meant bidding adieu to compiz (apart from the bling it does actually provide some pretty damned useful features). I thought I'd give it a shot at least for a while on principal, since I've always had to think of the nvidia drivers as a necessary evil, and their lack of proper xrandr support is really, really annoying.
Since then I haven't noticed any problems, apart from the following message after every suspend/resume:
[ 931.213239] Uhhuh. NMI received for unknown reason 80 on CPU 0.
[ 931.213240] You have some hardware problem, likely on the PCI bus.
[ 931.213242] Dazed and confused, but trying to continue
No idea what's causing it, but otherwise the system is stable and functional. If anyone knows what can be done about this, I'd appreciate hearing from you.
My graphics card, by the way:
01:00.0 VGA compatible controller: nVidia Corporation G72M [Quadro NVS 110M/GeForce Go 7300] (rev a1)
In the time since then, the nouveau driver has made its way into unstable, and
a lot of distros (ubuntu, fedora, ...) are even using it as the default driver.
So before going any further with the 3d/compiz stuff, I highly recommend
that you get the packaged version working first. This will not only make
sure that the basic driver works, but it will also make recovery/rollback
a lot easier. Since it's all packaged in the standard repositories,
it's not much harder than installing a few packages and editing a few
files... so come on give it a try 
Getting the 2d driver working in Debian sid
Really, this is so easy it's hardly worth writing up, but i'm running with
the -v flag this morning...
Make sure you don't have any reference to nvidia in /etc/modules or other
places that might cause it to be automatically loaded. Then remove the
proprietary glx bindings, which would otherwise interfere with the
nouveau driver.
sudo apt-get remove nvidia-glx
Next, you'll need to install a kernel that includes updated drm support. Normally this would require a kernel >= 2.6.33.1, but the debian kernel maintainers have graciously backported this code to the 2.6.32-4 kernel package. note this isn't the default kernel version in testing/unstable at the time of writing so you have to explicitly install it along with the nouveau driver.
sudo apt-get install linux-image-2.6.32-4-amd64 xserver-xorg-video-nouveau
If you are already running this kernel and using the proprietary nvidia drivers,
via module-assistant or otherwise, you /might/ need to uninstall them first. On my
system I left them installed in the old kernel version but did not install them on
the new kernel version.
Required xorg configuration
I don't believe that nouveau is picked by default if available at the time of writing, so you have to have the following in your xorg.conf:
sudo sh -c "cat > /etc/X11/xorg.conf" << EOF
Section "Device"
Identifier "nVidia Corporation G72M [Quadro NVS 110M/GeForce Go 7300]"
Driver "nouveau"
EndSection
EOF
This was all I needed to do, apart from rebooting into the new kernel. For
those who don't want to venture outside the packaging system, this is about
as far as you can go for the time being. Though really, if you're bothering
to read this far along, you're probably more interested in what comes next 
Getting 3d working (without voiding your distro warranty)
So I thought I'd give the 3d-accelerated gallium a go. Their wiki page certainly makes one think twice, but at this point I was seriously fiending to get my bling back (well, the compiz scale plugin, anyway) and figured it wouldn't hurt too much to try.
Via google and trawling around in various forums I found a number of
how-to documents for getting Gallium3d on debian/ubuntu systems,
but most of them recommended doing things in a way that wasn't
easily reversible and that could totally hose your system in a
not-so-hypothetical worst-case scenario. It's pretty hard to roll back a
make install done into /usr, for example, and subsequent package upgrades
can leave your system even worse off as they will probably clobber half
of what you just did, leaving the other half totally broken.
And if you have a laptop which requires the gui for networking
(i.e. network-manager), then fixing/rolling-back the changes is not
entirely trivial, since you'd likely have to re-download the various
drm/mesa/xorg packages.
So enter the tried and true unix utility stow (apt-get install
stow, of course). stow is a utility that lets you install software
into arbitrary folders, and then manage the system-wide installation of
the software via a farm of symlinks pointing at the installed location.
It's probably more well known among the sysadmin crowd than the desktop
user crowd, but it's incredibly useful for the task at hand as it will
let us easily install multiple copies of different software in a way
that's very easy to update and/or roll-back. RTFM stow(1) for more
information.
So, on to the juicy stuff...
Install needed software, remove problematic packages, setup, etc
At least on my system, I had to remove libdrm-dev to keep things from accidentally
linking against it instead of the updated version that we install later on below. There's
probably a way to override this with CFLAGS but I figure this is simpler.
sudo apt-get remove libdrm-dev
sudo apt-get install stow build-essential xorg-dev git-core libtool mesa-common-dev automake autoconf
mkdir ~/nouveau
Compile a linux-2.6.34 RC kernel
cd ~/nouveau
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
cd linux-2.6
git remote add -f nouveau git://anongit.freedesktop.org/nouveau/linux-2.6
git checkout -b nouveau-master nouveau/master
make menuconfig
Enable device drivers -> staging drivers -> nouveau. Make any other
changes you might want to make (I just took defaults for everything else).
make deb-pkg
The linux-firmware-free debian package ships some of the same stuff from the
generated linux-firmware-image package. I decided to opt for the upstream
firmware, though I'm not sure that this is necessary:
sudo apt-get remove linux-firmware-free
sudo dpkg -i ~/nouveau/linux-image*.deb ~/nouveau/linux-firmware-image*.deb
sudo update-grub
Install a copy of dri2proto
cd ~/nouveau
git clone git://anongit.freedesktop.org/xorg/proto/dri2proto
cd dri2proto
./autogen.sh --prefix=/usr/local/stow/dri2proto-20100428
sudo mkdir -p /usr/local/stow/dri2proto-20100428
sudo make install
sudo stow -d /usr/local/stow dri2proto-20100428
Install a cutting-edge version of libdrm
cd ~/nouveau
git clone git://anongit.freedesktop.org/git/mesa/drm/
cd drm
./autogen.sh --prefix=/usr/local/stow/drm-20100428 --disable-intel --disable-radeon --enable-nouveau-experimental-api
make
sudo mkdir -p /usr/local/stow/drm-20100428
sudo make install
sudo stow -d /usr/local/stow drm-20100428
Likewise for mesa
cd ~/nouveau
git clone git://anongit.freedesktop.org/git/mesa/mesa
cd mesa
./autogen.sh --prefix=/usr/local/stow/mesa-20100428 --enable-debug --enable-glx-tls --disable-asm --with-dri-drivers= --enable-gallium-nouveau --disable-gallium-intel --disable-gallium-radeon --disable-gallium-svga --with-state-trackers=glx,dri --with-demos=xdemos,demos,trivial,tests
make
sudo mkdir -p /usr/local/stow/mesa-20100428
sudo make install
sudo stow -d /usr/local/stow mesa-20100428
And now the xorg driver...
cd ~/nouveau
git clone git://anongit.freedesktop.org/git/nouveau/xf86-video-nouveau/
cd xf86-video-nouveau
./autogen.sh --prefix=/usr/local/stow/xf86-video-nouveau-20100428
make
sudo mkdir -p /usr/local/stow/xf86-video-nouveau-20100428
sudo make install
sudo stow -d /usr/local/stow xf86-video-nouveau-20100428
Configuration
Configure xorg to look in alternate location for xorg modules and set up DRI
sudo sh -c "cat > /etc/X11/xorg.conf" << EOF
Section "Files"
ModulePath "/usr/local/lib/xorg/modules,/usr/lib/xorg/modules"
EndSection
Section "ServerFlags"
Option "GlxVisuals" "all"
Endsection
Section "Device"
Identifier "nVidia Corporation G72M [Quadro NVS 110M/GeForce Go 7300]"
Driver "nouveau"
EndSection
EOF
I've /heard/ that disabling TV out is a necessity on some cards, though I haven't seen this myself:
sudo sh -c "cat > /etc/modprobe.d/nouveau.conf" << EOF
# not entirely sure this is necessary, but heard it helped with vsync
options nouveau tv_disable=1
EOF
Finally, the one thing that can not be done in /usr/local is making the
nouveau DRI driver available. From what I could tell the path that mesa
uses to find the DRI libraries is hard-coded (corrections welcome).
Since you won't already have this file (otherwise, why would you be
following along here?), just drop in a symlink to the file in /usr/local:
sudo ln -s /usr/local/lib/dri/nouveau_dri.so /usr/lib/dri/
Finally, reboot, and hope for the best 
Riding the bleeding edge: how to upgrade later on
Again, stow really shows its usefullness here. Using drm as an example case,
upgrading an individual component would look something like:
cd ~/nouveau/drm
git clean -dfx
git pull
today=`date +%Y%m%d`
./autogen.sh --prefix=/usr/local/stow/drm-$today --disable-intel --disable-radeon --enable-nouveau-experimental-api
make
sudo mkdir -p /usr/local/stow/drm-$today
sudo make install
sudo stow -d /usr/local/stow -D drm-20100428
sudo stow -d /usr/local/stow drm-$today
Note how the old version is still installed, so you can switch components around pretty freely.
Abort! Abort!: How to roll back changes
In case your system becomes totally unusable, or you've decided that you just want to go back to the safe and cozy world of what's provided by the debian packages, simply do the following:
- Remove the
ModulePathdirective from yourxorg.conf - Remove the
nouveau_dri.sosymlink from/usr/lib/dri - Possibly(?) boot into your previous kernel (I read grumblings about ABI breakage on lkml which might mean that you can't use the same libdrm with the old/new drivers, though I haven't had to find out).
Assuming that you're using the nouveau driver from unstable, this should be all that is necessary.
Conclusion
Overall impression
I am very, very happy with the resulting setup. I was led to believe that this whole thing was much more unstable and unusable than it actually is! While there are certainly some glitches and missing features, I feel comfortable enough using it that I don't have plans to switch back. But if I find myself in a situation where I can't use the new setup, rolling back shouldn't be a problem either. I figure I'm happy with what I have now and it will only get better.
What works, at least to a "kinda works" level.
- A standard compiz installation
- Basic 3d support
- Xvideo
- Display port switching with fn-F7 (but see below)
- Suspend/Resume (most of the time, anyway)
What still doesn't work
- Suspend/Resume doesn't /always/ work, I wouldn't call it 100% reliable anyway
- I've had an occasional graphical lockup of the system requiring reboot
- Switching display ports only partially works (one of the two displays will usually be corrupted, though the other will work ok).
- Docking and undocking combined with suspend/resume and switching displays seems to trigger some hard lockups.
- Haven't tried extensive use of 3d-apps yet.
- Graphics glitches/artifacts
- A number of things will make compiz crash. When this happens it automatically
falls back to metacity, so it's not a huge problem--just pretty annoying.
- gnome-display-properties
- using the compiz "Resize Window" plugin with "Normal" (change it to "Rectangle")
- switching from lcd to external monitor will sometimes cause it to crash
- I would guess other plugins might be problematic as well, though at least what I had enabled previously seems to work okay.
- Any kind of GPU power saving features (graphics card runs pretty hot, anyway).
Further reading, references
- DRM installation guide
- Kernel modesetting tips
- Gallium3D howto
- Tips for lockup recovery
- An earlier ubuntu how-to
Thanks
A couple shoutouts to those who helped me with this on the way:
- all the developers involved in the respective projects... a big Thank You!
- ymanton, mwk, calim, paerley on #nouveau for helping me troubleshoot kernel/libdrm issues.
- KiBi on #debian-x for all the work getting nouveau into unstable.
Feedback / Corrections
Unforunately, I've had to disable comments for the time being due to overwhelming quantities of spam. So please drop me a line via email or on IRC (seanius on oftc/freenode).