FedoraではAutoQAというテストフレームワークがあってそれで自動テストが走っているのでそれを試してみました。忘れそうなので自分用メモの意味で書いてるのでほとんど俺得なだけだと思いますが… もし試したい人はリンク先からAutoQAのWikiに行ってそこからインストール方法を見てください。
さて、試したのはrats_installっていうテストでRawhideのAcceptance Test PlanにあるInstallabilityのテストのモジュールです。AutoQAの内部ではフレームワークとしてLinuxカーネルのテストに使われてるAutotestを使ってます。rats_installの場合はインストールのテストなのでQEMU、QEMUを操作するためのlibvirtなども使っています。テストしたFedoraのバージョンは確実にテストが成功するはずのF13です。
autoqaの実行自体はこんな感じです。
[masami@moon]~/autoqa% sudo autoqa post-tree-compose http://ftp.riken.jp/Linux/fedora/releases/13/Fedora/x86_64/os/ --local
autoqaに渡している最初の引数"post-tree-compose"はhook名でhookとテストが紐付いてます。次はレポジトリのURLです。ここで指定するURLは普通にネットワークインストールするときと同じです。最後の「--local」はテストをスケジューリングするのではなくて、直接実行する場合につけます。存在しているhook名は引数なしでautoqaを実行すれば最後の行に表示されます。
[masami@moon]~/fedora-test% autoqa
Usage: autoqa HOOKNAME [options] ...
Options:
-h, --help show this help message (or hook help message if
HOOKNAME given) and exit
-a ARCH, --arch=ARCH arch to run the test(s) on; can be used multiple
times; by default 'noarch' arch is used
-t TEST, --test=TEST run only the given test(s) instead of all relevant
ones; can be used multiple times; if you specify a
test that wouldn't be run by default it will be forced
to run
--keep-control-file do not delete generated control files
--dryrun, --dry-run do not actually execute commands, just show what would
be done (implies --keep-control-file)
--local do not schedule jobs - run test(s) directly on the
local machine
-l, --list-tests list the tests for the given hookname - do not run any
tests
--autotest-server=AUTOTEST_SERVER
sets the autotest-server hostname used for creating
URLs to results;hostname of the local machine is used
by default
No hookname given.
Known hooks: post-koji-build post-bodhi-update post-tree-compose post-repo-updateそして、以下が実際のログです。インストールイメージのチェックをしたり、カーネルの起動チェック、Anacondaのチェック、パッケージのインストール、リブートなどやってるのが分かると思います。
[masami@moon]~/autoqa% sudo autoqa post-tree-compose http://ftp.riken.jp/Linux/fedora/releases/13/Fedora/x86_64/os/ --local [sudo] password for masami: Error: Can't evaluate test 'depcheck': [Errno 2] No such file or directory: '/usr/share/autotest/client/site_tests/depcheck/control.autoqa' Error: Can't evaluate test 'anaconda': [Errno 2] No such file or directory: '/usr/share/autotest/client/site_tests/anaconda/control.autoqa' 21:43:11 INFO | Writing results to /usr/share/autotest/client/results/post-tree-compose:rats_install.noarch 21:43:11 INFO | Initializing the state engine 21:43:11 DEBUG| Persistent state variable __steps now set to [] 21:43:11 INFO | Symlinking init scripts 21:43:11 DEBUG| Running 'grep :initdefault: /etc/inittab' 21:43:11 DEBUG| Running 'ln -sf /usr/share/autotest/client/tools/autotest /etc/init.d/autotest' 21:43:11 DEBUG| Running 'ln -sf /usr/share/autotest/client/tools/autotest /etc/rc5.d/S99autotest' 21:43:11 DEBUG| Dropping caches 21:43:11 DEBUG| Running 'sync' 21:43:11 DEBUG| Running 'echo 3 > /proc/sys/vm/drop_caches' 21:43:11 DEBUG| Running 'rpm -qa' 21:43:14 INFO | START ---- ---- timestamp=1284640994 localtime=Sep 16 21:43:14 21:43:14 DEBUG| Persistent state variable __group_level now set to 1 21:43:14 INFO | START rats_install rats_install timestamp=1284640994 localtime=Sep 16 21:43:14 21:43:14 DEBUG| Persistent state variable __group_level now set to 2 21:43:14 DEBUG| Crash handling system enabled 21:43:14 DEBUG| Running 'rpm -qa' 21:43:16 DEBUG| Running 'yum -y install python-fedora' 21:43:16 DEBUG| [stdout] Loaded plugins: presto, refresh-packagekit 21:43:17 DEBUG| [stdout] Setting up Install Process 21:43:18 DEBUG| [stdout] Package python-fedora-0.3.20-1.fc13.noarch already installed and latest version 21:43:18 DEBUG| [stdout] Nothing to do 21:43:18 DEBUG| Running 'yum -y install libvirt qemu-kvm python-virtinst pax' 21:43:18 DEBUG| [stdout] Loaded plugins: presto, refresh-packagekit 21:43:18 DEBUG| [stdout] Setting up Install Process 21:43:20 DEBUG| [stdout] Package libvirt-0.8.2-1.fc13.x86_64 already installed and latest version 21:43:20 DEBUG| [stdout] Package 2:qemu-kvm-0.12.5-1.fc13.x86_64 already installed and latest version 21:43:20 DEBUG| [stdout] Package python-virtinst-0.500.4-1.fc13.noarch already installed and latest version 21:43:20 DEBUG| [stdout] Package pax-3.4-10.fc12.x86_64 already installed and latest version 21:43:20 DEBUG| [stdout] Nothing to do 21:43:20 DEBUG| Running 'service libvirtd status' 21:43:20 DEBUG| [stdout] libvirtd (pid 1527) を実行中... 21:43:20 DEBUG| Running 'lokkit --disabled' 21:43:20 INFO | Test started. Number of iterations: 1 21:43:20 INFO | Executing iteration 1 of 1 21:43:20 INFO | Dropping caches between iterations 21:43:20 DEBUG| Running 'sync' 21:43:20 DEBUG| Running 'echo 3 > /proc/sys/vm/drop_caches' 21:43:20 DEBUG| Running './install.py -s /usr/share/autotest/client/tmp/tmpgF118L_rats_install -l /usr/share/autotest/client/results/post-tree-compose:rats_install.noarch/rats_install/results http://ftp.riken.jp/Linux/fedora/releases/13/Fedora/x86_64/os/ 2>&1' Retrieving file .treeinfo... | 2.4 kB 00:00 ... Retrieving file vmlinuz... | 6.7 MB 00:00 ... Retrieving file initrd.img... | 57 MB 00:00 ... Allocating 'RATS.img' | 8.0 GB 00:00 Creating domain... | 0 B 00:00 21:46:12 DEBUG| [stdout] 21:46:12 DEBUG| [stdout] Starting install... 21:46:12 DEBUG| [stdout] Domain installation still in progress. You can reconnect to 21:46:12 DEBUG| [stdout] the console to complete the installation process. 21:56:08 DEBUG| [stdout] Domain RATS destroyed 21:56:08 DEBUG| [stdout] 21:56:08 DEBUG| [stdout] Domain RATS has been undefined 21:56:08 DEBUG| [stdout] 21:56:08 DEBUG| [stdout] Vol /var/lib/libvirt/images/RATS.img deleted 21:56:08 DEBUG| [stdout] 21:56:08 DEBUG| [stdout] == installer image sanity test == 21:56:08 DEBUG| [stdout] CHECK: [general] section in .treeinfo: OK 21:56:08 DEBUG| [stdout] CHECK: arch = x86_64 in [general]: OK 21:56:08 DEBUG| [stdout] CHECK: [images-x86_64] section: OK 21:56:08 DEBUG| [stdout] CHECK: kernel: OK 21:56:08 DEBUG| [stdout] CHECK: initrd: OK 21:56:08 DEBUG| [stdout] CHECK: mainimage item in [stage2] section: OK 21:56:08 DEBUG| [stdout] stage2 mainimage: install.img 21:56:08 DEBUG| [stdout] CHECK: vmlinuz non-empty: OK 21:56:08 DEBUG| [stdout] CHECK: initrd.img non-empty: OK 21:56:08 DEBUG| [stdout] CHECK: boot.iso non-empty: OK 21:56:08 DEBUG| [stdout] CHECK: install.img non-empty: OK 21:56:08 DEBUG| [stdout] CHECK: [checksum] section: OK 21:56:08 DEBUG| [stdout] CHECK: boot.iso sha256: OK 21:56:08 DEBUG| [stdout] CHECK: vmlinuz sha256: OK 21:56:08 DEBUG| [stdout] CHECK: install.img sha256: OK 21:56:08 DEBUG| [stdout] CHECK: initrd.img sha256: OK 21:56:08 DEBUG| [stdout] sanity check complete. 21:56:08 DEBUG| [stdout] tree timestamp: Thu May 13 10:01:37 2010 (local time) 21:56:08 DEBUG| [stdout] TEST RESULT: images: OK 21:56:08 DEBUG| [stdout] prep: 21:56:08 DEBUG| [stdout] uncompressing initrd 21:56:08 DEBUG| [stdout] saving original initrd 21:56:08 DEBUG| [stdout] writing kickstart 21:56:08 DEBUG| [stdout] adding kickstart to initrd 21:56:08 DEBUG| [stdout] recompressing initrd 21:56:08 DEBUG| [stdout] new guest name: RATS 21:56:08 DEBUG| [stdout] starting virt guest 21:56:08 DEBUG| [stdout] 21:56:08 DEBUG| [stdout] == kernel boot test == 21:56:08 DEBUG| [stdout] opening serial console 21:56:08 DEBUG| [stdout] serial console at /dev/pts/2 21:56:08 DEBUG| [stdout] polling for boot messages 21:56:08 DEBUG| [stdout] kernel version 2.6.33.3-85.fc13.x86_64 booted, initrd started OK 21:56:08 DEBUG| [stdout] TEST RESULT: kernel_boot: OK 21:56:08 DEBUG| [stdout] 21:56:08 DEBUG| [stdout] == anaconda loading stage2 test == 21:56:08 DEBUG| [stdout] polling for stage2 startup message 21:56:08 DEBUG| [stdout] anaconda 13.42 stage2 has started 21:56:08 DEBUG| [stdout] TEST RESULT: load_stage2: OK 21:56:08 DEBUG| [stdout] 21:56:08 DEBUG| [stdout] == anaconda disk probe test == 21:56:08 DEBUG| [stdout] Starting minimon log monitor on port 52680 21:56:08 DEBUG| [stdout] anaconda now in step setuptime 21:56:08 DEBUG| [stdout] anaconda now in step autopartitionexecute 21:56:08 DEBUG| [stdout] anaconda now in step storagedone 21:56:08 DEBUG| [stdout] anaconda now in step enablefilesystems 21:56:08 DEBUG| [stdout] storage setup completed OK 21:56:08 DEBUG| [stdout] TEST RESULT: diskprobe: OK 21:56:08 DEBUG| [stdout] 21:56:08 DEBUG| [stdout] == anaconda package installation test == 21:56:08 DEBUG| [stdout] Waiting up to 20 minutes for package installation to complete 21:56:08 DEBUG| [stdout] anaconda now in step bootloadersetup 21:56:08 DEBUG| [stdout] anaconda now in step reposetup 21:56:08 DEBUG| [stdout] anaconda now in step basepkgsel 21:56:08 DEBUG| [stdout] anaconda now in step postselection 21:56:08 DEBUG| [stdout] anaconda now in step reipl 21:56:08 DEBUG| [stdout] anaconda now in step install 21:56:08 DEBUG| [stdout] anaconda now in step preinstallconfig 21:56:08 DEBUG| [stdout] anaconda now in step installpackages 21:56:08 DEBUG| [stdout] anaconda now in step postinstallconfig 21:56:08 DEBUG| [stdout] anaconda now in step writeconfig 21:56:08 DEBUG| [stdout] Package installation completed OK 21:56:08 DEBUG| [stdout] TEST RESULT: package_install: OK 21:56:08 DEBUG| [stdout] 21:56:08 DEBUG| [stdout] == anaconda bootloader setup test == 21:56:08 DEBUG| [stdout] Waiting for bootloader setup to complete 21:56:08 DEBUG| [stdout] anaconda now in step firstboot 21:56:08 DEBUG| [stdout] anaconda now in step instbootloader 21:56:08 DEBUG| [stdout] anaconda now in step writeksconfig 21:56:08 DEBUG| [stdout] Bootloader setup complete 21:56:08 DEBUG| [stdout] TEST RESULT: bootloader: OK 21:56:08 DEBUG| [stdout] 21:56:08 DEBUG| [stdout] == completing installation == 21:56:08 DEBUG| [stdout] Waiting for step 'dopostaction' 21:56:08 DEBUG| [stdout] anaconda now in step setfilecon 21:56:08 DEBUG| [stdout] anaconda now in step copylogs 21:56:08 DEBUG| [stdout] anaconda now in step methodcomplete 21:56:08 DEBUG| [stdout] anaconda now in step postscripts 21:56:08 DEBUG| [stdout] anaconda now in step dopostaction 21:56:08 DEBUG| [stdout] Install completed in 595 seconds 21:56:08 DEBUG| [stdout] TEST RESULT: install_complete: OK 21:56:08 DEBUG| [stdout] 21:56:08 DEBUG| [stdout] == testing complete == 21:56:08 DEBUG| [stdout] Cleaning up guest. 21:56:08 INFO | Failed to send results to irb: ServerError(http://localhost/get_trees, 404, Not Found) 21:56:08 INFO | Test finished after 1 iterations. 21:56:08 DEBUG| Logging subprocess finished 21:56:08 DEBUG| Logging subprocess finished 21:56:08 DEBUG| Running 'grep MemTotal /proc/meminfo' 21:56:08 DEBUG| Running 'rpm -qa' 21:56:11 INFO | GOOD rats_install rats_install timestamp=1284641771 localtime=Sep 16 21:56:11 completed successfully 21:56:11 DEBUG| Persistent state variable __group_level now set to 1 21:56:11 INFO | END GOOD rats_install rats_install timestamp=1284641771 localtime=Sep 16 21:56:11 21:56:11 DEBUG| Dropping caches 21:56:11 DEBUG| Running 'sync' 21:56:11 DEBUG| Running 'echo 3 > /proc/sys/vm/drop_caches' 21:56:11 DEBUG| Persistent state variable __group_level now set to 0 21:56:11 INFO | END GOOD ---- ---- timestamp=1284641771 localtime=Sep 16 21:56:11 21:56:11 DEBUG| Logging subprocess finished 21:56:11 DEBUG| Logging subprocess finished
このテストはkickstartを使うのでks.cfgにレポジトリを書いたり、ソースコードのコメントからkickstart内でリブートさせると上手くいかなさそうだったのでリブートさせないようにしてます。
あとrats_install.pyにimport文を追加しました。これって最近のcommitで削除された文なんですけども、これがないとconfig_loader()が無いって怒られてテストの最後の処理で失敗してたので。
diff --git a/tests/rats_install/ks.cfg b/tests/rats_install/ks.cfg index 0fa5954..5b8244c 100644 --- a/tests/rats_install/ks.cfg +++ b/tests/rats_install/ks.cfg @@ -1,6 +1,6 @@ # Basic kickstart file for RATS install -url --url=REAL_URL_GOES_HERE +url --url=http://ftp.riken.jp/Linux/fedora/releases/13/Fedora/x86_64/os/ lang en_US.UTF-8 keyboard us network --device eth0 --mtu=1500 --bootproto dhcp @@ -12,7 +12,7 @@ zerombr clearpart --all --initlabel autopart bootloader --location=mbr -reboot +#reboot # packages should be a superset of critpath %packages diff --git a/tests/rats_install/rats_install.py b/tests/rats_install/rats_install.py index 21c95de..fcd11a2 100644 --- a/tests/rats_install/rats_install.py +++ b/tests/rats_install/rats_install.py @@ -25,6 +25,7 @@ from autotest_lib.client.bin import utils from autotest_lib.client.common_lib import error from autoqa.test import AutoQATest from autoqa.decorators import ExceptionCatcher +from autotest_lib.client.bin.test_config import config_loader class rats_install(AutoQATest): version = 1
