RSS Atom Add a new post titled:

Update: Closely followed by 0.5.4 to fix an embarassing brown paper bag bug:

  • Correct argument handling for system-status command

Get it from gitorious or http://www.hellion.org.uk/qcontrol/releases/0.5.4/.

I've just released qcontrol 0.5.3. Changes since the last release:

  • Reduce spaminess of temperature control (Debian bug #727150).
  • Support for enabling/disabling RTC on ts219 and ts41x. Patch from Michael Stapelberg (Debian bug #732768).
  • Support for Synology Diskstation and Rackstation NASes. Patch from Ben Peddell.
  • Return correct result from direct command invocation (Debian bug #617439).
  • Fix ts41x LCD detection.
  • Improved command line argument parsing.
  • Lots of internal refactoring and cleanups.

Get it from gitorious or http://www.hellion.org.uk/qcontrol/releases/0.5.3/.

The Debian package will be uploaded shortly.

Posted Fri 11 Apr 2014 16:04:01 BST Tags:

At the Mini-debconf in Cambridge back in November there was an ARM Sprint (which Hector wrote up as a Bits from ARM porters mail). During this there a brief discussion about using GRUB as a standard bootloader, particularly for ARM server devices. This has the advantage of providing a more "normal" (which in practice means "x86 server-like") as well as flexible solution compared with the existing flash-kernel tool which is often used on ARM.

On ARMv7 devices this will more than likely involve chain loading from the U-Boot supplied by the manufacturer. For test and development it would be useful to be able to set up a similar configuration using Qemu.

Cross-compilers

Although this can be built and run on an ARM system I am using a cross compiler here. I'm using gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux from Linaro, which can be downloaded from the linaro-toolchain-binaries page on Launchpad. (It looks like 2013.10 is the latest available right now, I can't see any reason why that wouldn't be fine).

Once the cross-compiler has been downloaded unpack it somewhere, I will refer to the resulting gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux directory as $CROSSROOT.

Make sure $CROSSROOT/bin (which contains arm-linux-gnueabihf-gcc etc) is in your $PATH.

Qemu

I'm using the version packaged in Jessie, which is 1.7.0+dfsg-2. We need both qemu-system-arm for running the final system and qemu-user to run some of the tools. I'd previously tried an older version of qemu (1.6.x?) and had some troubles, although they may have been of my own making...

Das U-boot for Qemu

First thing to do is to build a suitable u-boot for use in the qemu emulated environment. Since we need to make some configuration changes we need to build from scratch.

Start by cloning the upstream git tree:

$ git clone git://git.denx.de/u-boot.git
$ cd u-boot

I am working on top of e03c76c30342 "powerpc/mpc85xx: Update CONFIG_SYS_FSL_TBCLK_DIV for T1040" dated Wed Dec 11 12:49:13 2013 +0530.

We are going to use the Versatile Express Cortex-A9 u-boot but first we need to enable some additional configuration options:

  • CONFIG_API -- This enables the u-boot API which Grub uses to access the lowlevel services provided by u-boot. This means that grub doesn't need to contains dozens of platform specific flash, mmc, nand, network, console drivers etc and can be completely platform agnostic.
  • CONFIG_SYS_MMC_MAX_DEVICE -- Setting CONFIG_API needs this.
  • CONFIG_CMD_EXT2 -- Useful for accessing EXT2 formatted filesystems. In this example I use a VFAT /boot for convenience but in a real system we would want to use EXT2 (or even something more modern)).
  • CONFIG_CMD_ECHO -- Just useful.

You can add all these to include/configs/vexpress_common.h:

#define CONFIG_API
#define CONFIG_SYS_MMC_MAX_DEVICE   1
#define CONFIG_CMD_EXT2
#define CONFIG_CMD_ECHO

Or you can apply the patch which I sent upstream:

$ wget -O - http://patchwork.ozlabs.org/patch/304786/raw | git apply --index
$ git commit -m "Additional options for grub-on-uboot"

Finally we can build u-boot:

$ make CROSS_COMPILE=arm-linux-gnueabihf- vexpress_ca9x4_config
$ make CROSS_COMPILE=arm-linux-gnueabihf-

The result is a u-boot binary which we can load with qemu.

GRUB for ARM

Next we can build grub. Start by cloning the upstream git tree:

$ git clone git://git.sv.gnu.org/grub.git
$ cd grub

By default grub is built for systems which have RAM at address 0x00000000. However the Versatile Express platform which we are targeting has RAM starting from 0x60000000 so we need to make a couple of modifications. First in grub-core/Makefile.core.def we need to change arm_uboot_ldflags, from:

-Wl,-Ttext=0x08000000

to

-Wl,-Ttext=0x68000000

and second we need make a similar change to include/grub/offsets.h changing GRUB_KERNEL_ARM_UBOOT_LINK_ADDR from 0x08000000 to 0x68000000.

Now we are ready to build grub:

$ ./autogen.sh
$ ./configure --host arm-linux-gnueabihf
$ make

Now we need to build the final grub "kernel" image, normally this would be taken care of by grub-install but because we are cross building grub we cannot use this and have to use grub-mkimage directly. However the version we have just built is for the ARM target and not for host we are building things on. I've not yet figured out how to build grub for ARM while building the tools for the host system (I'm sure it is possible somehow...). Luckily we can use qemu to run the ARM binary:

$ cat load.cfg
set prefix=(hd0)
$ qemu-arm -r 3.11 -L $CROSSROOT/arm-linux-gnueabihf/libc \
    ./grub-mkimage -c load.cfg -O arm-uboot -o core.img -d grub-core/ \
    fat ext2 probe terminal scsi ls linux elf msdospart normal help echo

Here we create load.cfg which is the setup script which will be built into the grub kernel, our version just sets the root device so that grub can find the rest of its configuration.

Then we use qemu-arm-static to invoke grub-mkimage. The "-r 3.11" option tells qemu to pretend to be a 3.11 kernel (which is required by the libc used by our cross compiler, without this you will get a fatal: kernel too old message) and "-L $CROSSROOT/..." tells it where to find the basic libraries, such as the dynamic linker (luckily grub-mkimage doesn't need much in the way of libraries so we don't need a full cross library environment.

The grub-mkimage command passes in the load.cfg and requests an output kernel targeting arm-uboot, core.img is the output file and the modules are in grub-core (because we didn't actually install grub in the target system, normally these would be found in /boot/grub). Lastly we pass in a list of default modules to build into the kernel, including filesystem drivers (fat, ext2), disk drivers (scsi), partition handling (msdos), loaders (linux, elf), the menu system (normal) and various other bits and bobs.

So after all the we now have our grub kernel in core.img.

Putting it all together

Before we can launch qemu we need to create various disk images.

Firstly we need some images for the 2 64M flash devices:

$ dd if=/dev/zero of=pflash0.img bs=1M count=64
$ dd if=/dev/zero of=pflash1.img bs=1M count=64

We will initialise these later from the u-boot command line.

Secondly we need an image for the root filesystem on an MMC device. I'm using a FAT formatted image here simply for the convenience of using mtools to update the images during development.

$ dd if=/dev/zero of=mmc.img bs=1M count=16
$ /sbin/mkfs.vfat mmc.img

Thirdly we need a kernel, device tree and grub configuration on our root filesystem. For the first two I extracted them from the standard armmp kernel flavour package. I used the backports.org version 3.11-0.bpo.2-armmp version and extracted /boot/vmlinuz-3.11-0.bpo.2-armmp as vmlinuz and /usr/lib/linux-image-3.11-0.bpo.2-armmp/vexpress-v2p-ca9.dtb as dtb. Then I hand coded a simple grub.cfg:

menuentry 'Linux' {
        echo "Loading vmlinuz"
        set root='hd0'
        linux /vmlinuz console=ttyAMA0 ro debug
        devicetree /dtb
}

In a real system the kernel and dtb would be provided by the kernel packages and grub.cfg would be generated by update-grub.

Now that we have all the bits we need copy them into the root of mmc.img. Since we are using a FAT formatted image we can use mcopy from the mtools package.

$ mcopy -v -o -n -i mmc.img core.img dtb vmlinuz grub.cfg ::

Finally after all that we can run qemu passing it our u-boot binary and the mmc and flash images and requesting a Cortex-A9 based Versatile Express system with 1GB of RAM:

$ qemu-system-arm -M vexpress-a9 -kernel u-boot -m 1024m -sd mmc.img \
    -nographic -pflash pflash0.img -pflash pflash1.img

Then at the VExpress# prompt we can configure the default bootcmd to load grub and save the environment to the flash images. The backslash escapes (\$ and \;) should be included as written here so that e.g. the variables are only evaluated when bootcmd is evaluated and not immediately when setting bootcmd and the bootm is set as part of bootcmd instead of executed immediately:

VExpress# setenv bootcmd fatload mmc 0:0 \${loadaddr} core.img \; bootm \${loadaddr}
VExpress# saveenv

Now whenever we boot the system it will automatically load boot grub from the mmc and launch it. Grub in turn will load the Linux binary and DTB and launch those. I haven't actually configure Linux with a root filesystem here so it will eventually panic after failing to find root.

Future work

The most pressing issue is the hard coded load address built in to the grub kernel image. This is something which needs to be discussed with the upstream grub maintainers as well as the Debian package maintainers.

Now that the ARM packages have hit Debian (in experimental in the 2.02~beta2-1 package) I also plan to start looking at debian-installer integration as well as updating flash-kernel to setup the chain load of grub instead of loading a kernel directly.

Posted Thu 26 Dec 2013 13:20:51 GMT Tags:

Until now qcontrol has mostly only supported only ARM (kirkwood) based devices (upstream has a configuration example for the HP Media Vault too, but I don't know if it is used).

Debian bug #712191 asked for at least some basic support for x86 based devices. The mostly don't use the QNAP PIC used on the ARM devices so much of the qcontrol functionality is irrelevant but at least some of them do have a compatible A125 LCD.

Unfortunately I don't have any x86 QNAP devices and I've been unable to figure out a way to detect that we are running on an QNAP box as opposed to any random x86 box so I've not been able to implement the hardware auto-detection used on ARM to configure qcontrol for the appropriate device at installation time. I don't want to include a default configuration which tries to drive an LCD on some random serial port since I have no way of knowing what will be on the other end or what the device might do if sent random bytes of the LCD control protocol.

So I've implemented debconf prompting for the device type which is used only if auto-detection fails, so it shouldn't change anything for existing users on ARM. You can find this in version 0.5.2-3~exp1 in experimental (see DebianExperimental on the Debian wiki for how to use experimental).

Currently the package only knows about the existing set of ARM platforms and a default "unknown" platform, which has an empty configuration. If you have a QNAP device (ARM or x86) which is not currently supported then please install the package from experimental and tailor /etc/qcontrol.conf for you platform (e.g. by uncommenting the a125 support and giving it the correct serial port). Then send me the result along with the device's name. If the device is an ARM one please also send me the contents of /proc/cpuinfo too so I can implement auto-detection.

If you know how to detect a particular x86 QNAP device programmatically (via DMI decoding, PCI probing, sysfs etc, but make sure it is 100% safe on non-QNAP platforms) then please do let me know.

Posted Sat 05 Oct 2013 12:22:49 BST Tags:

I've just released qcontrol 0.5.2. Changes since the last release:

  • Fix memory corruption bug (Debian bug #708376).
  • Ignore code 0x43 from PIC, avoid spamming log.
  • Implement hysteresis for fan speed (Arno via Debian bug #709095).
  • Support for systemd socket activation and added systemd service files (both from Michael Stapelberg).

Get it from gitorious or http://www.hellion.org.uk/qcontrol/releases/0.5.2/.

The Debian package will be uploaded shortly.

Posted Sun 29 Sep 2013 19:02:11 BST Tags:

Going to DebConf13!

Seems like DebConf 13 is in full swing, I'm not heading over until tomorrow because I spent the weekend at the Bloodstock Festival. As sad as I am to miss the Cheese and Wine party tonight, having seen a dozen or so excellent bands since Friday, lubricated by a few gallons of Hobgoblin, and rounded off by Exodus, Anthrax & Slayer last night I think I made the right call!

So I got home around lunchtime today, unpacked my rucksack, scrubbed myself clean, repacked again for Switzerland (no tent this time). Next up is a curry with my parents tonight and then catching the Eurostar tomorrow morning, I'll have plenty of time for hacking on my cubieboard2 on the train and finally arrive in Vaumarcus tomorrow evening. See everyone soon!

Posted Mon 12 Aug 2013 15:14:05 BST Tags:

Quite a while ago at a Xen hackathon (I think the Cambridge one in 2011, has it really been 2 years?) the topic of bug tracking came up.

The developers who were present (myself included) expressed a general dislike for web based bug trackers, and in particular the existing bugzilla wasn't working for us for a variety of reasons (primarily lack of resource for care and general tending of it). They also liked the existing work flow based around mails to the Xen mailing lists, there was a feeling that this fits well with the demographics of the bugs which get reported to Xen: a sizable number of the issues are configuration errors or misunderstandings which can usually quickly be dealt with through a small email thread (followed by a request to please update the wiki). The project also gets quite a few bug reports which are initially "unactionable" due to of lack logs or a coherent description, in these cases a little bit of back and forth can quickly determine whether the reporter is going to provide the necessary additional information (this is something we are also trying to fix with clearer guidance and an ongoing effort to provide a reportbug-like tool for Xen).

However there were concerns that bigger issues which couldn't be dealt with quickly were falling through the cracks. We kicked around a few ideas for how a bug tracker that met our needs might work and that was mostly that.

Since then I've kicked the ideas around in my head a bit and searched for an existing system which would fit what was required. In the end I didn't find anything which quite fitted the bill and about six months ago I was looking for an interesting hacking project (perhaps something a bit different to my day job, hypervisor and kernel hacking) so I decided to bite the bullet, dust of my rusty Perl skills and hack something together.

Yes, that's right, I've written Yet Another Bug Tracker(tm). I confess I feel a bit dirty for doing so.

I've taken a fair bit of inspiration from debbugs (the Debian bug tracker) but I've turned the concept a little on its head. Debbugs is effectively instantiating a new mailing list for each bug report (each NNNNNN@bugs.debian.org can be thought of as a mailing list plus some additional bug specific metadata). Instead I implemented a model which takes an existing mailing list and acts as a list archive which provides a mechanism to track individual threads (or subthreads) as bugs (each bug is effectively a root Message-Id plus a little bit of metadata).

In common with debbugs bugs are created and subsequently manipulated by sending mail to a special control@ address. Commands are available to create bugs as well as to graft and prune subthreads onto existing bugs (to cope with people who break threading or offtopic subthreads etc). There is also a read-only web interface for browsing the bugs. Internally the system is subscribed to the list and records details of each message and their relationships (e.g. References headers etc) in a sqlite database. A set of CGI scripts (using the Perl CGI module) provide an interface to browse bugs.

Since the system tracks bugs based on mailing list threads I've named it Emesinae which is a "thread-legged bug" (yes, I spent longer than is strictly necessary browsing entomological websites). I've made the code available on gitorious under a GPLv2 license, see https://gitorious.org/emesinae.

I announced an alpha instance of Emesinae back in May which tracks the xen-devel mailing list. You can find the bug tracker at http://bugs.xenproject.org/xen/. Since then I've been slowly fleshing out some more useful features (such as tagging of affected branches).

It's not seen a great deal of uptake from the Xen developers yet but there are already some bugs being tracked and I'm hopeful that it will prove useful in the long run. In any case it was a fun hacking project for a few rainy evenings (and a couple more train and plane journeys).

Posted Wed 03 Jul 2013 14:41:36 BST Tags:

I've just released qcontrol 0.5.1. Changes since the last release:

  • Add build targets to enable static linking (Original patch by Frans Pop).
  • Wake-on-Lan and EUP control (by Michael Stapelberg, Debian bug #703888).
  • Updated example configurations (based on Debian package).
  • Support loading configuration snippets from a directory (Ian Campbell, Debian bug #697574).
  • Various other bug fixes.

I also put together a very basic homepage.

Get it from gitorious or http://www.hellion.org.uk/qcontrol/releases/0.5.1/.

The Debian package will be uploaded shortly.

Posted Sun 12 May 2013 19:28:28 BST Tags:

Taking over Qcontrol upstream

When I took over the qcontrol package in Debian, back in October 2012, I was aware that upstream hadn't been active for quite a while (since late 2009). I figured that would be OK since the software was pretty simple and pretty much feature complete.

Nevertheless since I've been maintaining the package I've had a small number of wishlist bugs (many with patches, thanks guys!) which are really upstream issues. While I could carry them as patches in the Debian package they would really be better taken care of by an upstream.

With that in mind I mailed the previous upstream (and original author) Byron Bradley back in February to ask if he would mind if I took over as upstream. Unfortunately I haven't yet heard back, given how long upstream has been inactive for this isn't terribly surprising so having waited several months I have decided to just go ahead and take over upstream development. Thanks Byron for all your work on the project, I hope you don't mind me taking over.

Since it is unlikely that I will be able to access the old website or Subversion repository I have converted the repository to git and uploaded it to gitorious:

I don't expect I'll be doing an awful lot of proactive upstream development but at least now I have a hat I can put on when someone reports an "upstream" issue against qcontrol and somewhere which can accept upstream patches from myself and others.

I'm still deciding what to do about a website etc. I may just enable the gitorious wiki or I may setup an ikiwiki software site type setup, which has the advantage of being a little more flexible and providing a neat way of dealing with bug reports without being too much overhead to setup up.

In the meantime its been almost 5 years since the last release, so...

New Release: qcontrol 0.5.0

This is a rollup of some changes which were made in the old upstream SVN repository but never released and some patches which had been made in the Debian packaging. What's here corresponds to the Debian 0.4.2+svn-r40-3 package.

The 0.4.2 release was untagged in SVN, but corresponds to r14, new stuff since then includes:

  • Support for more hardware (TS-119, TS-219, TS-41x, all by Martin Michlmayr)
  • Support auto power on feature (by Martin Michlmayr)
  • Support for the A125 LCD on TS-419P devices (by Bernhard R. Link)
  • Support for a daemon mode, including syslog (Byron Bradley)
  • Support for disabling the watchdog on TS-219P II and TS-419P II (Helmut Pozimski)
  • Various other fixes (Loïc Minier, Martin, Bernhard, Byron)

As well as the change of maintainer I think the addition of the daemon mode warrants the bump to 0.5.0.

Get the new release from git or http://www.hellion.org.uk/qcontrol/releases/0.5.0/.

Posted Sat 04 May 2013 16:23:22 BST Tags:

Working with virtualisation I find myself occasionally needing to generate a random MAC address for use with a virtual machine. I have a stupid little local script which spits one out, which is all fine and dandy for my purposes. However this week one of my colleagues was writing a wiki page and needed to include instructions on how to set the MAC address on the Arndale board. He wanted to include a shell snippet to generate one and I suggested that there must be websites which will generate you a suitable address. But when I came to look we found that not a single one of the half a dozen site which I looked at handled the locally administered or multicast bits correctly, meaning that they would randomly generate either multicast addresses or addresses using assigned OUI (or both). Using a random OUI may not cause too much trouble in practice but using a multicast address is sure to lead to strange behaviour, and in any case it's kind of lame.

So last night I sat down and wrote a quick hack cgi script to generate random addresses and tonight I deployed it at: http://www.hellion.org.uk/cgi-bin/randmac.pl.

This is where the world and his dog leaves a comment pointing me to the existing service my searches missed...

Posted Fri 26 Apr 2013 21:49:57 BST Tags:

I've just got back from Linaro Connect Asia 2013 in Hong Kong. This was my first time back in Hong Kong since 1995 when I left after living there for 10 years. It was really interesting to see how much the place had changed, as well as how much it hadn't!

I started the week by giving a talk together with Stefano Stabellini Introducing Xen on ARM to a full house. The slides can be found on the event page or google docs. There is a video on youtube but due to the cameras being setup for a fish-bowl style panel it is unfortunately mostly of my backside. The reception was excellent and the talk seeded a large number of hallway track conversations with all the many folks who are interested in Xen on the ARM platform.

The day after our talk Lars Kurth (Xen.org community manager) and Mark Heath (VP of XenServer Engineering, and my boss) gave a keynote (video) announced Citrix's intention to join the Linaro Enterprise Group (AKA "LEG") on the 1st of April. This is pretty exciting for me since it means I'll get to work much more closely with the Linaro and ARM communities and spend even more of my time hacking on Xen on ARM!

At some point Stefano and myself were also video interviewed by Charbax of http://armdevices.net/ but it doesn't appear to have gone online yet. Lars and Mark also gave interviews which are online already.

One nice thing about Connect is that the afternoons are left free of talks so I managed to find time in the hackrooms to implement SMP host support and 64-bit domain 0 support for Xen on ARM as well as getting multiarch cross building working for upstream Xen with Wookey's kind assistance (no patches for this yet since I still need to clean them up). It was also useful to catch up with the ARM kernel developers and KVM guys to see what they are up to and share ideas etc.

Finally right at the end Stefano and myself showed Xen on ARM at Demo Friday. We had Xen running on the Arndale Board, ARM Chromebook and 64-bit ARMv8 using the emulators (no hardware being available yet).

All in all I had an excellent (if tiring) week, there was loads of interest in Xen on ARM and in Citrix joining LEG and everyone was very welcoming. I'm really looking forward to the next Connect in Dublin.

Outside of Connect I managed to find time to visit Sham Shui Po at look at all the gadgets but managed to resist buying anything (Stefano got a Thinkpad X230, which was enough retail therapy for both of us!). I caught up with my sister (who lives there again), her boyfriend and my parents (who happened to be visiting her that week). I also managed have some drinks with some old school friends, most of whom I've not seen for nearly twenty years, but it was just like old times ;-).

I'm off again to Hong Kong next week with my family to visit my sister (yes, there was some bad planning involved here...). This time I plan to do all the toursity type stuff that I never did while I lived there. I've also got tickets for the Hong Kong Rubgy Sevens and hopefully I'll manage catch up with some more old friends. I probably won't be able to resist buying gadgets this time around though.

Posted Wed 13 Mar 2013 14:19:19 GMT Tags:

This blog is powered by ikiwiki.