ThermostatQA
============

This project contains QA tests for Thermostat. Thermostat itself
contains a set of unit and integration tests, but this tries to be
much more comprehensive.

Requirements
-----------

gnome-keyring
mongodb
xdotool
import (ImageMagick)
lynx

Running
-------

Build and run using:

    $ make

This will download any necessary tools and then set up everything and run the
entire test suite. Since the test exercises the GUI, it might be a good idea to
run it in a seprate VNC session.

Directory Structure
-------------------

- `bin/` compiled classes
- `logs/`  logs from tests
- `reports/` reports from tests
- `screenshots/` screenshots from tests
- `scripts/` scripts for running the tsts

Important makefile targets
--------------------------

all             builds project an runs tests

build           builds all classes

runtests        executes all test (automaticaly builds project and download
                necessary software if needed)

listtests       lists all available tests

<testName>      runs single test of that name
                
run-api-test    run test which checks api compatibility of tested thermostat
                against "other" thermostat using Java API Compliance Checker
                (also see THERMOSTAT_OTHER_* in "Important makefile variables"
                section)

report          generates report from tests (runtests is needed to be
                executed before)

javadoc         generages javadoc


thermostat      clones, builds and sets-up thermostat (if necessary)

tomcat          downloads and unpacks apache-tomcat (if necessary)


Important makefile variables
----------------------------

BUILD_DIR           : directory genrated by build phase used to store compiled 
                      classes and also downloaded files/software
                      and custom configuration directories used for testing
                      (default: bin)

CLASSES_DIR         : directory where generated classes are stored
                      (default: <BUILD_DIR>/classes)

THERMOSTAT_REPO_URL : repository used to clone Thermostat for testing
                      (default: http://icedtea.classpath.org/hg/thermostat/ )

THERMOSTAT_VERSION  : version for which ThermostatQA is configured. It is also
					  used to refer tested thermostat in api compatibility
					  report.
                      (default: 1.1.0)

THERMOSTAT_USER_DIR : directory used for thermostat user configuration during testing
                      (used to set USER_THERMOSTAT_HOME environment variable)
                      (default: <BUILD_DIR>/.thermostat)

THERMOSTAT_DIR      : directory with thermostat for testing (if present)
                      or directory where to clone thermostat (if not present)
                      (default: <BUILD_DIR>/thermostat)

THERMOSTAT_OTHER_*	: same as THERMOSTAT_* but refers to thermostat against which
					  compatibility is tested.
					  (defaults:
					  THERMOSTAT_OTHER_DIR:		<BUILD_DIR>/thermostat-other
					  THERMOSTAT_OTHER_VERSION:	1.0
					  THERMOSTAT__OTHER_REPO_URL:
					  http://icedtea.classpath.org/hg/release/thermostat-1.0/)

TOMCAT_DIR          : directory with apache-tomcat used for testing (if present)
                      or directory where to download thermostat (if not present)
                      (default: <BUILD_DIR>/tomcat)

KILL_DAEMONS        : if this variable is set to 1 test suite will not ask for
                      confirmation before killing daemons (mongo, gnome-keyring).
                      if there is any of them running.
                      (see Side Efects)

SCREENSHOT_SOURCE   : sets backend for making screenshots and for mouse,
                      keyboard emulation, possible values:
                      robot  - uses java.awt.Robot class
                      xvfb   - uses import command (imagemagick) to make
                               screenshots and xdotool for mouse move and
                               keyboard, it may be necessary when testing inside
                               vnc or xvfb (imagemagick and xdotool need to be
                               installed first to use this backend)
                      (default: robot)

Examples
---------

testing using default config (builds tests, downloads tomcat and all nessessary
software and runs test using default config)

    make    

testing thermostat from custom repository (url):

    make THERMOSTAT_REPO_URL=<someUrl>

testing thermostat stored at custom path, using custom tomcat: 

    make THERMOSTAT_DIR=<somePath>  TOMCAT_DIR=<somePath>
    
testing api compatibility of tested thermostat against "other" version:

	make run-api-test THERMOSTAT_REPO_URL=<someUrl> \
	THERMOSTAT_OTHER_REPO_URL=<someOtherUrl> \
	THERMOSTAT_OTHER_VERSION=<someVersion>

same as previous but using previously downloaded thermostats:

	make run-api-test THERMOSTAT_DIR=<somePath> \
	THERMOSTAT_OTHER_DIR=<someOtherPath> \
	THERMOSTAT_OTHER_VERSION=<someVersion>

(urls and paths can also be combined)


Side Effects
------------

By default these tests do not modify settings in ~/.thermostat or any other
files or settings outside this (ThermostatQA) directory (and subdirs).

When using custom Thermostat, tests also modify files in
<THERMOSTAT_DIR>/distribution/target/image/etc directory.

When using custom apache-tomcat ThermostatQA also deploys thermostat war archive
in <TOMCAT_DIR>/webapps/thermostat.

Running tests needs to terminate running instances of mongod
and gnome-keyring-daemon, because it runs theese daemons with custom
configuration. If some of these deamons running test suite will ask
for confirmation to kill them. This can be skiped by setting KILL_DAEMONS
makefile variable.
(see Important makefile variables)