読者です 読者をやめる 読者になる 読者になる

Fedora AutoQAを動かす

fedora

FedoraではAutoQAというテストフレームワークがあってそれで自動テストが走っているのでそれを試してみました。忘れそうなので自分用メモの意味で書いてるのでほとんど俺得なだけだと思いますが… もし試したい人はリンク先からAutoQAのWikiに行ってそこからインストール方法を見てください。
さて、試したのはrats_installっていうテストでRawhideのAcceptance Test PlanにあるInstallabilityのテストのモジュールです。AutoQAの内部ではフレームワークとしてLinuxカーネルのテストに使われてるAutotestを使ってます。rats_installの場合はインストールのテストなのでQEMUQEMUを操作するためのlibvirtなども使っています。テストしたFedoraのバージョンは確実にテストが成功するはずのF13です。

autoqaの実行自体はこんな感じです。

[masami@moon]~/autoqa% sudo autoqa post-tree-compose http://ftp.riken.jp/Linux/fedora/releases/13/Fedora/x86_64/os/  --local

autoqaに渡している最初の引数"post-tree-compose"はhook名でhookとテストが紐付いてます。次はレポジトリのURLです。ここで指定するURLは普通にネットワークインストールするときと同じです。最後の「--local」はテストをスケジューリングするのではなくて、直接実行する場合につけます。存在しているhook名は引数なしでautoqaを実行すれば最後の行に表示されます。

[masami@moon]~/fedora-test% autoqa
Usage: autoqa HOOKNAME [options] ...

Options:
  -h, --help            show this help message (or hook help message if
                        HOOKNAME given) and exit
  -a ARCH, --arch=ARCH  arch to run the test(s) on; can be used multiple
                        times; by default 'noarch' arch is used
  -t TEST, --test=TEST  run only the given test(s) instead of all relevant
                        ones; can be used multiple times; if you specify a
                        test that wouldn't be run by default it will be forced
                        to run
  --keep-control-file   do not delete generated control files
  --dryrun, --dry-run   do not actually execute commands, just show what would
                        be done (implies --keep-control-file)
  --local               do not schedule jobs - run test(s) directly on the
                        local machine
  -l, --list-tests      list the tests for the given hookname - do not run any
                        tests
  --autotest-server=AUTOTEST_SERVER
                        sets the autotest-server hostname used for creating
                        URLs to results;hostname of the local machine is used
                        by default
No hookname given.
Known hooks: post-koji-build post-bodhi-update post-tree-compose post-repo-update

そして、以下が実際のログです。インストールイメージのチェックをしたり、カーネルの起動チェック、Anacondaのチェック、パッケージのインストール、リブートなどやってるのが分かると思います。

[masami@moon]~/autoqa% sudo autoqa post-tree-compose http://ftp.riken.jp/Linux/fedora/releases/13/Fedora/x86_64/os/  --local
[sudo] password for masami: 
Error: Can't evaluate test 'depcheck': [Errno 2] No such file or directory: '/usr/share/autotest/client/site_tests/depcheck/control.autoqa'
Error: Can't evaluate test 'anaconda': [Errno 2] No such file or directory: '/usr/share/autotest/client/site_tests/anaconda/control.autoqa'
21:43:11 INFO | Writing results to /usr/share/autotest/client/results/post-tree-compose:rats_install.noarch
21:43:11 INFO | Initializing the state engine
21:43:11 DEBUG| Persistent state variable __steps now set to []
21:43:11 INFO | Symlinking init scripts
21:43:11 DEBUG| Running 'grep :initdefault: /etc/inittab'
21:43:11 DEBUG| Running 'ln -sf /usr/share/autotest/client/tools/autotest /etc/init.d/autotest'
21:43:11 DEBUG| Running 'ln -sf /usr/share/autotest/client/tools/autotest /etc/rc5.d/S99autotest'
21:43:11 DEBUG| Dropping caches
21:43:11 DEBUG| Running 'sync'
21:43:11 DEBUG| Running 'echo 3 > /proc/sys/vm/drop_caches'
21:43:11 DEBUG| Running 'rpm -qa'
21:43:14 INFO | START	----	----	timestamp=1284640994	localtime=Sep 16 21:43:14	
21:43:14 DEBUG| Persistent state variable __group_level now set to 1
21:43:14 INFO | 	START	rats_install	rats_install	timestamp=1284640994	localtime=Sep 16 21:43:14	
21:43:14 DEBUG| Persistent state variable __group_level now set to 2
21:43:14 DEBUG| Crash handling system enabled
21:43:14 DEBUG| Running 'rpm -qa'
21:43:16 DEBUG| Running 'yum -y install python-fedora'
21:43:16 DEBUG| [stdout] Loaded plugins: presto, refresh-packagekit
21:43:17 DEBUG| [stdout] Setting up Install Process
21:43:18 DEBUG| [stdout] Package python-fedora-0.3.20-1.fc13.noarch already installed and latest version
21:43:18 DEBUG| [stdout] Nothing to do
21:43:18 DEBUG| Running 'yum -y install libvirt qemu-kvm python-virtinst pax'
21:43:18 DEBUG| [stdout] Loaded plugins: presto, refresh-packagekit
21:43:18 DEBUG| [stdout] Setting up Install Process
21:43:20 DEBUG| [stdout] Package libvirt-0.8.2-1.fc13.x86_64 already installed and latest version
21:43:20 DEBUG| [stdout] Package 2:qemu-kvm-0.12.5-1.fc13.x86_64 already installed and latest version
21:43:20 DEBUG| [stdout] Package python-virtinst-0.500.4-1.fc13.noarch already installed and latest version
21:43:20 DEBUG| [stdout] Package pax-3.4-10.fc12.x86_64 already installed and latest version
21:43:20 DEBUG| [stdout] Nothing to do
21:43:20 DEBUG| Running 'service libvirtd status'
21:43:20 DEBUG| [stdout] libvirtd (pid  1527) を実行中...
21:43:20 DEBUG| Running 'lokkit --disabled'
21:43:20 INFO | Test started. Number of iterations: 1
21:43:20 INFO | Executing iteration 1 of 1
21:43:20 INFO | Dropping caches between iterations
21:43:20 DEBUG| Running 'sync'
21:43:20 DEBUG| Running 'echo 3 > /proc/sys/vm/drop_caches'
21:43:20 DEBUG| Running './install.py -s /usr/share/autotest/client/tmp/tmpgF118L_rats_install -l /usr/share/autotest/client/results/post-tree-compose:rats_install.noarch/rats_install/results http://ftp.riken.jp/Linux/fedora/releases/13/Fedora/x86_64/os/ 2>&1'
Retrieving file .treeinfo...                             | 2.4 kB     00:00 ... 
Retrieving file vmlinuz...                               | 6.7 MB     00:00 ... 
Retrieving file initrd.img...                            |  57 MB     00:00 ... 
Allocating 'RATS.img'                                    | 8.0 GB     00:00     
Creating domain...                                       |    0 B     00:00     
21:46:12 DEBUG| [stdout] 
21:46:12 DEBUG| [stdout] Starting install...
21:46:12 DEBUG| [stdout] Domain installation still in progress. You can reconnect to 
21:46:12 DEBUG| [stdout] the console to complete the installation process.
21:56:08 DEBUG| [stdout] Domain RATS destroyed
21:56:08 DEBUG| [stdout] 
21:56:08 DEBUG| [stdout] Domain RATS has been undefined
21:56:08 DEBUG| [stdout] 
21:56:08 DEBUG| [stdout] Vol /var/lib/libvirt/images/RATS.img deleted
21:56:08 DEBUG| [stdout] 
21:56:08 DEBUG| [stdout] == installer image sanity test ==
21:56:08 DEBUG| [stdout] CHECK: [general] section in .treeinfo: OK
21:56:08 DEBUG| [stdout] CHECK: arch = x86_64 in [general]: OK
21:56:08 DEBUG| [stdout] CHECK: [images-x86_64] section: OK
21:56:08 DEBUG| [stdout] CHECK: kernel: OK
21:56:08 DEBUG| [stdout] CHECK: initrd: OK
21:56:08 DEBUG| [stdout] CHECK: mainimage item in [stage2] section: OK
21:56:08 DEBUG| [stdout] stage2 mainimage: install.img
21:56:08 DEBUG| [stdout] CHECK: vmlinuz non-empty: OK
21:56:08 DEBUG| [stdout] CHECK: initrd.img non-empty: OK
21:56:08 DEBUG| [stdout] CHECK: boot.iso non-empty: OK
21:56:08 DEBUG| [stdout] CHECK: install.img non-empty: OK
21:56:08 DEBUG| [stdout] CHECK: [checksum] section: OK
21:56:08 DEBUG| [stdout] CHECK: boot.iso sha256: OK
21:56:08 DEBUG| [stdout] CHECK: vmlinuz sha256: OK
21:56:08 DEBUG| [stdout] CHECK: install.img sha256: OK
21:56:08 DEBUG| [stdout] CHECK: initrd.img sha256: OK
21:56:08 DEBUG| [stdout] sanity check complete.
21:56:08 DEBUG| [stdout] tree timestamp: Thu May 13 10:01:37 2010 (local time)
21:56:08 DEBUG| [stdout] TEST RESULT: images: OK
21:56:08 DEBUG| [stdout] prep:
21:56:08 DEBUG| [stdout]   uncompressing initrd
21:56:08 DEBUG| [stdout]   saving original initrd
21:56:08 DEBUG| [stdout]   writing kickstart
21:56:08 DEBUG| [stdout]   adding kickstart to initrd
21:56:08 DEBUG| [stdout]   recompressing initrd
21:56:08 DEBUG| [stdout]   new guest name: RATS
21:56:08 DEBUG| [stdout] starting virt guest
21:56:08 DEBUG| [stdout] 
21:56:08 DEBUG| [stdout] == kernel boot test ==
21:56:08 DEBUG| [stdout] opening serial console
21:56:08 DEBUG| [stdout]   serial console at /dev/pts/2
21:56:08 DEBUG| [stdout] polling for boot messages
21:56:08 DEBUG| [stdout] kernel version 2.6.33.3-85.fc13.x86_64 booted, initrd started OK
21:56:08 DEBUG| [stdout] TEST RESULT: kernel_boot: OK
21:56:08 DEBUG| [stdout] 
21:56:08 DEBUG| [stdout] == anaconda loading stage2 test ==
21:56:08 DEBUG| [stdout] polling for stage2 startup message
21:56:08 DEBUG| [stdout] anaconda 13.42 stage2 has started
21:56:08 DEBUG| [stdout] TEST RESULT: load_stage2: OK
21:56:08 DEBUG| [stdout] 
21:56:08 DEBUG| [stdout] == anaconda disk probe test ==
21:56:08 DEBUG| [stdout] Starting minimon log monitor on port 52680
21:56:08 DEBUG| [stdout]   anaconda now in step setuptime
21:56:08 DEBUG| [stdout]   anaconda now in step autopartitionexecute
21:56:08 DEBUG| [stdout]   anaconda now in step storagedone
21:56:08 DEBUG| [stdout]   anaconda now in step enablefilesystems
21:56:08 DEBUG| [stdout] storage setup completed OK
21:56:08 DEBUG| [stdout] TEST RESULT: diskprobe: OK
21:56:08 DEBUG| [stdout] 
21:56:08 DEBUG| [stdout] == anaconda package installation test ==
21:56:08 DEBUG| [stdout] Waiting up to 20 minutes for package installation to complete
21:56:08 DEBUG| [stdout]   anaconda now in step bootloadersetup
21:56:08 DEBUG| [stdout]   anaconda now in step reposetup
21:56:08 DEBUG| [stdout]   anaconda now in step basepkgsel
21:56:08 DEBUG| [stdout]   anaconda now in step postselection
21:56:08 DEBUG| [stdout]   anaconda now in step reipl
21:56:08 DEBUG| [stdout]   anaconda now in step install
21:56:08 DEBUG| [stdout]   anaconda now in step preinstallconfig
21:56:08 DEBUG| [stdout]   anaconda now in step installpackages
21:56:08 DEBUG| [stdout]   anaconda now in step postinstallconfig
21:56:08 DEBUG| [stdout]   anaconda now in step writeconfig
21:56:08 DEBUG| [stdout] Package installation completed OK
21:56:08 DEBUG| [stdout] TEST RESULT: package_install: OK
21:56:08 DEBUG| [stdout] 
21:56:08 DEBUG| [stdout] == anaconda bootloader setup test ==
21:56:08 DEBUG| [stdout] Waiting for bootloader setup to complete
21:56:08 DEBUG| [stdout]   anaconda now in step firstboot
21:56:08 DEBUG| [stdout]   anaconda now in step instbootloader
21:56:08 DEBUG| [stdout]   anaconda now in step writeksconfig
21:56:08 DEBUG| [stdout] Bootloader setup complete
21:56:08 DEBUG| [stdout] TEST RESULT: bootloader: OK
21:56:08 DEBUG| [stdout] 
21:56:08 DEBUG| [stdout] == completing installation ==
21:56:08 DEBUG| [stdout] Waiting for step 'dopostaction'
21:56:08 DEBUG| [stdout]   anaconda now in step setfilecon
21:56:08 DEBUG| [stdout]   anaconda now in step copylogs
21:56:08 DEBUG| [stdout]   anaconda now in step methodcomplete
21:56:08 DEBUG| [stdout]   anaconda now in step postscripts
21:56:08 DEBUG| [stdout]   anaconda now in step dopostaction
21:56:08 DEBUG| [stdout] Install completed in 595 seconds
21:56:08 DEBUG| [stdout] TEST RESULT: install_complete: OK
21:56:08 DEBUG| [stdout] 
21:56:08 DEBUG| [stdout] == testing complete ==
21:56:08 DEBUG| [stdout] Cleaning up guest.
21:56:08 INFO | Failed to send results to irb: ServerError(http://localhost/get_trees, 404, Not Found)
21:56:08 INFO | Test finished after 1 iterations.
21:56:08 DEBUG| Logging subprocess finished
21:56:08 DEBUG| Logging subprocess finished
21:56:08 DEBUG| Running 'grep MemTotal /proc/meminfo'
21:56:08 DEBUG| Running 'rpm -qa'
21:56:11 INFO | 		GOOD	rats_install	rats_install	timestamp=1284641771	localtime=Sep 16 21:56:11	completed successfully
21:56:11 DEBUG| Persistent state variable __group_level now set to 1
21:56:11 INFO | 	END GOOD	rats_install	rats_install	timestamp=1284641771	localtime=Sep 16 21:56:11	
21:56:11 DEBUG| Dropping caches
21:56:11 DEBUG| Running 'sync'
21:56:11 DEBUG| Running 'echo 3 > /proc/sys/vm/drop_caches'
21:56:11 DEBUG| Persistent state variable __group_level now set to 0
21:56:11 INFO | END GOOD	----	----	timestamp=1284641771	localtime=Sep 16 21:56:11	
21:56:11 DEBUG| Logging subprocess finished
21:56:11 DEBUG| Logging subprocess finished

このテストはkickstartを使うのでks.cfgにレポジトリを書いたり、ソースコードのコメントからkickstart内でリブートさせると上手くいかなさそうだったのでリブートさせないようにしてます。
あとrats_install.pyにimport文を追加しました。これって最近のcommitで削除された文なんですけども、これがないとconfig_loader()が無いって怒られてテストの最後の処理で失敗してたので。

diff --git a/tests/rats_install/ks.cfg b/tests/rats_install/ks.cfg
index 0fa5954..5b8244c 100644
--- a/tests/rats_install/ks.cfg
+++ b/tests/rats_install/ks.cfg
@@ -1,6 +1,6 @@
 # Basic kickstart file for RATS
 install
-url --url=REAL_URL_GOES_HERE
+url --url=http://ftp.riken.jp/Linux/fedora/releases/13/Fedora/x86_64/os/
 lang en_US.UTF-8
 keyboard us
 network --device eth0 --mtu=1500 --bootproto dhcp
@@ -12,7 +12,7 @@ zerombr
 clearpart --all --initlabel
 autopart
 bootloader --location=mbr
-reboot
+#reboot
 
 # packages should be a superset of critpath
 %packages
diff --git a/tests/rats_install/rats_install.py b/tests/rats_install/rats_install.py
index 21c95de..fcd11a2 100644
--- a/tests/rats_install/rats_install.py
+++ b/tests/rats_install/rats_install.py
@@ -25,6 +25,7 @@ from autotest_lib.client.bin import utils
 from autotest_lib.client.common_lib import error
 from autoqa.test import AutoQATest
 from autoqa.decorators import ExceptionCatcher
+from autotest_lib.client.bin.test_config import config_loader
 
 class rats_install(AutoQATest):
     version = 1