01 July 2013

468. Kernel 3.10 on Debian

NOTE I: As of 3.10.2 the nvidia module will still not build.  I've also tried 3.10.5 and it also does not work.

NOTE II: I'm getting random slowdowns on my SL410 laptop with intel graphics. Not sure if it's the same issue as this: http://verahill.blogspot.com.au/2013/03/368-slow-mouse-and-keyboard-triggered.html
Once kworker shows up in top everything grinds to a slow crawl. I also notice that I never used 3.9 on that laptop, so the issue may be present there too.

There are several ways of building a kernel. The easiest (a purely subjective statement) is to use kernel-package i.e. make-kpkg. However, every now and again I see people writing that it's been deprecated.

Either way, start by doing
sudo apt-get install fakeroot build-essential ncurses-dev
mkdir ~/tmp
cd ~/tmp
wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.10.tar.xz
tar xvf linux-3.10.tar.xz
cd linux-3.10/
cat /boot/config-`uname -r`>.config
make oldconfig
Timer tick handling 1. Periodic timer ticks (constant rate, no dynticks) (HZ_PERIODIC) (NEW) > 2. Idle dynticks system (tickless idle) (NO_HZ_IDLE) (NEW) 3. Full dynticks system (tickless) (NO_HZ_FULL) (NEW) choice[1-3]: Memory placement aware NUMA scheduler (NUMA_BALANCING) [N/y/?] (NEW) Simple CPU accounting cgroup subsystem (CGROUP_CPUACCT) [N/y/?] (NEW) Group CPU scheduler (CGROUP_SCHED) [N/y/?] (NEW) Automatic process group scheduling (SCHED_AUTOGROUP) [N/y/?] (NEW) Choose SLAB allocator 1. SLAB (SLAB) (NEW) > 2. SLUB (Unqueued Allocator) (SLUB) choice[1-2?]: Linux guest support (HYPERVISOR_GUEST) [N/y/?] (NEW) Timer frequency 1. 100 HZ (HZ_100) 2. 250 HZ (HZ_250) (NEW) 3. 300 HZ (HZ_300) > 4. 1000 HZ (HZ_1000) choice[1-4?]: Memory Hotplug (ACPI_HOTPLUG_MEMORY) [N/y/?] (NEW) AMD frequency sensitivity feedback powersave bias (X86_AMD_FREQ_SENSITIVITY) [N/m/?] (NEW) Kernel support for scripts starting with #! (BINFMT_SCRIPT) [Y/n/m/?] (NEW) InfiniBand media type support (TIPC_MEDIA_IB) [N/y/?] (NEW) Network Coding (BATMAN_ADV_NC) [N/y/?] (NEW) NETLINK: mmaped IO (NETLINK_MMAP) [N/y/?] (NEW) NETLINK: socket monitoring interface (NETLINK_DIAG) [N/m/y/?] (NEW) Dummy IRQ handler (DUMMY_IRQ) [N/m/y/?] (NEW) Generic on-chip SRAM driver (SRAM) [N/y/?] (NEW) ME Enabled Intel Chipsets (INTEL_MEI_ME) [N/m/y/?] (NEW) Block device as cache (BCACHE) [N/m/y/?] (NEW) Qualcomm Atheros AR816x/AR817x support (ALX) [N/m/y/?] (NEW) QLOGIC QLCNIC 83XX family SR-IOV Support (QLCNIC_SRIOV) [Y/n/?] (NEW) Realtek RTL8152 Based USB 2.0 Ethernet Adapters (USB_RTL8152) [N/m/?] (NEW) Atheros ath9k rate control (ATH9K_LEGACY_RATE_CONTROL) [N/y/?] (NEW) rt2800usb - Include support for rt55xx devices (EXPERIMENTAL) (RT2800USB_RT55XX) [N/y/?] (NEW) y Realtek RTL8188EE Wireless Network Adapter (RTL8188EE) [N/m/?] (NEW) IMS Passenger Control Unit driver (INPUT_IMS_PCU) [N/m/?] (NEW) Qualcomm Single-wire Serial Bus Interface (SSBI) (SSBI) [N/m/y/?] (NEW) Analog Devices ADT7310/ADT7320 (SENSORS_ADT7310) [N/m/y/?] (NEW) National Semiconductor LM95234 (SENSORS_LM95234) [N/m/?] (NEW) Nuvoton NCT6775F and compatibles (SENSORS_NCT6775) [N/m/y/?] (NEW) generic cpu cooling support (CPU_THERMAL) [N/y/?] (NEW) y ChromeOS Embedded Controller (MFD_CROS_EC) [N/m/y/?] (NEW) Silicon Laboratories 4761/64/68 AM/FM radio. (MFD_SI476X_CORE) [N/m/?] (NEW) System Controller Register R/W Based on Regmap (MFD_SYSCON) [N/y/?] (NEW) TI TPS65912 Power Management chip (MFD_TPS65912) [N/y/?] (NEW) Conexant cx25821 support (VIDEO_CX25821) [N/m/?] (NEW) Cypress firmware helper routines (CYPRESS_FIRMWARE) [N/m] (NEW) QXL virtual GPU (DRM_QXL) [N/m/?] (NEW) Apple infrared receiver (HID_APPLEIR) [N/m/?] (NEW) Enable USB persist by default (USB_DEFAULT_PERSIST) [Y/n/?] (NEW) USB-Wishbone adapter interface driver (USB_SERIAL_WISHBONE) [N/m/?] (NEW) USB Physical Layer drivers (USB_PHY) [N/y/?] (NEW) PXA 27x (USB_PXA27X) [N/m/?] (NEW) MARVELL PXA2128 USB 3.0 controller (USB_MV_U3D) [N/m/?] (NEW) LED Support for TI LP5562 LED driver chip (LEDS_LP5562) [N/m/?] (NEW) LED Camera Flash/Torch Trigger (LEDS_TRIGGER_CAMERA) [N/m/y/?] (NEW) y iSCSI Extentions for RDMA (iSER) target support (INFINIBAND_ISERT) [N/m/?] (NEW) Set system time from RTC on startup and resume (RTC_HCTOSYS) [Y/n/?] (NEW) Set the RTC time based on NTP synchronization (RTC_SYSTOHC) [Y/n/?] (NEW) RTC used to set the system time (RTC_HCTOSYS_DEVICE) [rtc0] (NEW) WIS GO7007 MPEG encoder support (VIDEO_GO7007) [N/m/?] (NEW) DesignWare USB2 DRD Core Support (USB_DWC2) [N/m/?] (NEW) pvpanic device support (PVPANIC) [N/m/y/?] (NEW) Reset Controller Support (RESET_CONTROLLER) [N/y/?] (NEW) XFS Verbose Warnings (XFS_WARN) [N/y/?] (NEW) Btrfs will run sanity tests upon loading (BTRFS_FS_RUN_SANITY_TESTS) [N/y/?] (NEW) Btrfs debugging support (BTRFS_DEBUG) [N/y/?] (NEW) EFI Variable filesystem (EFIVAR_FS) [N/m/y/?] (NEW) torture tests for RCU (RCU_TORTURE_TEST) [N/m/y/?] (NEW) Ring buffer startup self test (RING_BUFFER_STARTUP_TEST) [N/y/?] (NEW) Test functions located in the string_helpers module at runtime (TEST_STRING_HELPERS) [N/m/y] (NEW) CMAC support (CRYPTO_CMAC) [N/m/y/?] (NEW) SHA256 digest algorithm (SSSE3/AVX/AVX2) (CRYPTO_SHA256_SSSE3) [N/m/y/?] (NEW) y SHA512 digest algorithm (SSSE3/AVX/AVX2) (CRYPTO_SHA512_SSSE3) [N/m/y/?] (NEW) y Camellia cipher algorithm (x86_64/AES-NI/AVX2) (CRYPTO_CAMELLIA_AESNI_AVX2_X86_64) [N/m/y/?] (NEW) m Serpent cipher algorithm (x86_64/AVX2) (CRYPTO_SERPENT_AVX2_X86_64) [N/m/y/?] (NEW) m KVM legacy PCI device assignment support (KVM_DEVICE_ASSIGNMENT) [Y/n/?] (NEW) VHOST_SCSI TCM fabric driver (VHOST_SCSI) [N/m/?] (NEW)
make menuconfig

You can now enable any additional modules by navigating the menu structure. Note that most likely you don't have to enable anything in this step, but it can come in handy if there's a major transition (e.g. the way multimedia was handled changed between kernel 3.5 and 3.6) or if you want to enable a previously disabled option.

Then pick either method 1 or 2 below.

If you only want to compile modules that are currently in use (not a good idea if you want to use the same kernel on a range of computers, or have USB devices that aren't currently plugged in) you can do that by using make localmodconfig instead of make oldconfig. I wouldn't recommend it -- in most cases it won't make a faster kernel, and space and memory tends not to be much of in the way of issues these days.


Method 1. kernel-package
Below, change -j2 to -jX, where X is the number of cores in your CPU (not cores+1 or anything funny like that. See other posts on this blog for compilation performance tests)

sudo apt-get install kernel-package
make-kpkg clean
time fakeroot make-kpkg -j2 --initrd kernel_image kernel_headers
sudo dpkg -i ../linux-image-3.10.0_3.10.0-10.00.Custom_amd64.deb ../linux-headers-3.10.0_3.10.0-10.00.Custom_amd64.deb


Took 49 minutes on a 3-core AMD Athlon II, and used ca 7 Gb.

The files are shown below:
-rw-r--r--  1 me me 8.4M Jul  1 16:02 linux-headers-3.10.0_3.10.0-10.00.Custom_amd64.deb
-rw-r--r--  1 me me  32M Jul  1 16:00 linux-image-3.10.0_3.10.0-10.00.Custom_amd64.deb


Method 2. make deb-pkg

make clean
time make deb-pkg -j2
sudo dpkg -i ../linux-firmware-image_3.10.0-2_amd64.deb ../linux-headers-3.10.0_3.10.0-2_amd64.deb ../linux-libc-dev_3.10.0-2_amd64.deb ../linux-image-3.10.0_3.10.0-2_amd64.deb

It took ca 50 minutes, it used ca 7 Gb, and generated the following files:
-rw-r--r-- 1 me me 1.1M Jul  1 16:58 ../linux-firmware-image_3.10.0-2_amd64.deb
-rw-r--r-- 1 me me 9.7M Jul  1 16:58 ../linux-headers-3.10.0_3.10.0-2_amd64.deb
-rw-r--r-- 1 me me 458M Jul  1 17:13 ../linux-image-3.10.0_3.10.0-2_amd64.deb
-rw-r--r-- 1 me me 920K Jul  1 16:58 ../linux-libc-dev_3.10.0-2_amd64.deb

Note the size of the linux-image-3.10.0_3.10.0-2_amd64.deb package.


Difference:
There are a few differences. One method, the kpgk one, is supposedly 'deprecated', but it's been working fine in the past and will work fine in the future (at least for some time).

The make dev-pkg method also generates a much bigger image file -- by a factor of 15 or so.

Otherwise, the chief difference, from what I can see, is if you want to uninstall the kernel. Remove the libc-dev package requires you to downgrade the package using apt-get, by specifying a version e.g.

apt-cache policy linux-libc-dev
linux-libc-dev: Installed: 3.10.0-1 Candidate: 3.10.0-1 Version table: *** 3.10.0-1 0 100 /var/lib/dpkg/status 3.9.6-1~bpo70+1 0 100 http://ftp.iinet.net.au/debian/debian/ wheezy-backports/main amd64 Packages 100 http://ftp.debian.org/debian/ wheezy-backports/main amd64 Packages 3.2.46-1 0 500 http://ftp.iinet.net.au/debian/debian/ wheezy/main amd64 Packages 3.2.41-2+deb7u2 0 500 http://security.debian.org/ wheezy/updates/main amd64 Packages
sudo apt-get install linux-libc-dev=3.9.6-1~bpo70+1

I personally prefer the kernel-package approach.

10 comments:

  1. After following the kernel-package directions Debian is still unable to build nvidia modules.

    The packaged kernel 3.10 at

    http://packages.debian.org/experimental/allpackages

    cannot be installed because of a dependency on linux-kbuild of this version - so it cannot be compared.

    Ubuntu Saucy runs its packaged Nvidia with its packaged kernel 3.10 perfectly well on this same computer - but the newly stable 3.10 kernel still does not compile modules for Debian. Its performance can only be appreciated on a laptop with intel graphics.

    The Ubuntu developers

    https://launchpad.net/~xorg-edgers/+archive/ppa

    have had for a few weeks now a 3.10 kernel running perfectly well with an nvidia patch which seems to be this:

    https://devtalk.nvidia.com/default/topic/543728/linux/building-nvidia-driver-on-kernel-3-9-0/3/

    However the following error recurs for a Debian build of kernel 3.10:

    var/lib/dkms/nvidia/325.08/build/nv-i2c.c:252:14: error: void value not ignored as it ought to be

    This was using the latest Nvidia installer from

    http://www.nvidia.com/object/unix.html

    instead of smxi - which is generally great but also did not provide an Nvidia compatible with 3.10.

    The directions on the Nvidia site to patch the installer do produce a custom.run file but pointing the patch at the files and the declared success of the patch does not produce modules for 3.10.

    ReplyDelete
    Replies
    1. How to get kernel 3.10 to compile Nvidia modules in at least Debian:

      Found this

      http://forums.gentoo.org/viewtopic-p-7339302.html

      which contained a link to this:

      https://devtalk.nvidia.com/default/topic/549208/linux/patch-for-325-08-on-linux-3-10/

      Using a different method of patching from the above two links -

      Download NVIDIA-Linux-x86_64-325.08.run from

      http://www.nvidia.com/object/unix.html

      and 3.10.patch.txt from

      https://devtalk.nvidia.com/default/topic/549208/linux/patch-for-325-08-on-linux-3-10/

      sh ./NVIDIA-Linux-x86_64-325.08.run --apply-patch 3.10.patch.txt

      produces a larger installer file NVIDIA-Linux-x86_64-325.08-custom.run - but in order for the patch to succeed it needs to be pointed at the directory of the two relevant files which is named kernel:

      /nvidia325patch$ sh ./NVIDIA-Linux-x86_64-325.08.run --apply-patch 3.10.patch.txt
      Verifying archive integrity... OK
      Uncompressing NVIDIA Accelerated Graphics Driver for Linux-x86_64 325.08.....................................................................................................................................................................
      can't find file to patch at input line 4
      Perhaps you used the wrong -p or --strip option?
      The text leading up to this was:
      --------------------------
      |diff -U 3 -H -d -r -N -- nv-i2c.c.orig nv-i2c.c
      |--- nv-i2c.c.orig 2013-07-01 21:45:45.241886356 -0400
      |+++ nv-i2c.c 2013-07-01 21:38:49.532484245 -0400
      --------------------------
      File to patch: kernel/nv-i2c.c
      patching file kernel/nv-i2c.c
      can't find file to patch at input line 32
      Perhaps you used the wrong -p or --strip option?
      The text leading up to this was:
      --------------------------
      |diff -U 3 -H -d -r -N -- nv-procfs.c.orig nv-procfs.c
      |--- nv-procfs.c.orig 2013-07-01 21:45:41.035242479 -0400
      |+++ nv-procfs.c 2013-07-01 21:38:49.532484245 -0400
      --------------------------
      File to patch: kernel/nv-procfs.c
      patching file kernel/nv-procfs.c
      Adding files to archive named "NVIDIA-Linux-x86_64-325.08-custom.run"...

      []

      Self-extractible archive "NVIDIA-Linux-x86_64-325.08-custom.run" successfully created.

      In order to install in Debian text-only environment - killall gdm3 and

      sh ./NVIDIA-Linux-x86_64-325.08-custom.run

      There is no difficulty in changing between manually and smxi installed nvidia or even packaged nvidia - smxi takes care of the details. It appears to be necessary to have a version change between installations or else unpatched files may be left behind.

      The 3.10 kernel performs well in gnome without problems mentioned in http://rglinuxtech.com/?p=773 and seems not only compatible but fast with virtualbox.

      As the 3.11 kernel seems almost ready to appear, 3.10 may benefit from a future patch from

      http://users.on.net/~ckolivas/kernel/

      Delete
    2. Hi Owen,
      thanks for the updates. I haven't had time to look closer at the nvidia + 3.10 issue, so I appreciate that you share your travails.

      It does seem like the patch didn't quite work above though -- have you looked at the rejected patches and compared with the sources? Given that these patches are often one-liners, it may be easy to patch by hand (and then generate your own patch from that).

      Delete
  2. Great work..... thank you.
    I have been reading your blog for a long time and wanted to let you know that I appreciate your work.

    Thanks,

    Arctic

    ReplyDelete
    Replies
    1. Thank you for the kind words. They are much appreciated. There's currently around 1,000 hits per day on this blog, but it's always difficult to know whether anyone other than myself actually get something out of this blog.

      Delete
  3. Thanks for the post; I'm running into the same problem and good to know that I'm not alone.

    I've been trying to get this to work using a vanilla 3.10.5 kernel on Debian Wheezy, but no luck and not very much information out there.

    Have you (or anybody reading these) had any luck compiling 3.10.5 + nVidia and getting things up and running? If so, please let us know.

    Thank you!

    ReplyDelete
    Replies
    1. Owen a few comments above has provided a few links to patches. I think success has been pretty mixed though.

      Looking at this page it seems that smxi might be patching for 3.10 now: http://smxi.org/sg/sgfxi
      There's a thread here: http://techpatterns.com/forums/about2308.html

      I haven't tried myself though. Given that 3.10 is the new long term support kernel hopefully nvidia get their act together an roll out a compatible driver.

      Delete
  4. Kernel 3.11-rc4 is fast and stable on intel i5 - when compiled for the processor. Presumably the AMD customization will be great too.

    For some reason it seems slow to tell Gnome about Kingston and other flash usb drives.

    Nvidia 325.15 on 3.11-rc4 is fine with the patch from Milos_SD -

    https://devtalk.nvidia.com/default/topic/549532/linux/linux-3-10-solved-3-11-solved-incompatibility-in-function-lsquo-nv_i2c_del_adapter-rsquo-error/2/

    The above patch is by far the smallest for the Nvidia 3.10 and 3.11 kernel problems so far, apparently addressing what to do after a check for KERNEL_VERSION.

    ReplyDelete
  5. "The make dev-pkg method also generates a much bigger image file -- by a factor of 15 or so."

    This results in a sensible kernel size:

    make deb-pkg INSTALL_MOD_STRIP=1

    http://unix.stackexchange.com/questions/30345/why-is-my-initial-ramdisk-so-big

    http://forums.debian.net/viewtopic.php?f=5&t=82222

    "Otherwise, the chief difference, from what I can see, is if you want to uninstall the kernel. Remove the libc-dev package requires you to downgrade the package using apt-get, by specifying a version"

    The libc-dev package seems to cause no problems if left uninstalled - and Debian's default libc-dev used instead.

    This is a great blog and a tremendous resource for Linux users - and no doubt Scientists.

    ReplyDelete
    Replies
    1. Again, thank you for the kind words and thank you for the feedback, suggestions and links.

      Delete