TestComplete integration into TeamCity
We use TestComplete for GUI testing in our work. For faster tests execution we use several PC’s with TestComplete. A unique part of full task list runs on each PC.
But how can we split task for maximum workload of PC? How to combine results without manual work?
It also would be good to include GUI testing in to continuous integration process. As a continuous integration server we use TeamCity. TestComplete can be simply integrated into native AutomatedQA continuous integration server. But this server doesn’t fit our needs. It’s payware and doesn’t support pre-tested commit.
We need to create this:
TeamCity runs GUI tests on all TestComplete clients. NUnit runner will be used as a program which starts distributes tests. All test results will be combining in TestFixture ctor. All TestComplete PC’s has a small web service to handle TeamCity request on test.
Let’s start. On all TestComplete PC we run web service. TeamCity contains project that contains Nunit TestFixture for distributed test run. Web service runs application by client’s parameters. Message sequence on initialization and run test.
As a dispatcher algorithm we use “round robin”. We need simply thread manager for controlling active thread count and providing simple algorithm of synchronization.
using System;
using System.Collections.Generic;
using System.Threading;
namespace ThreadLib {
///interface for thread result
public interface IThreadResult { }
public delegate T FuncDelegate<T>() where T : IThreadResult;//wrapper for delegate
public class ThreadControler<T> where T : IThreadResult {
private class ThreadParam {
public string Id;
public FuncDelegate<T> Delegate;
}
///delegates for execution
private readonly Dictionary<string, FuncDelegate<T>> Delegates = new Dictionary<string, FuncDelegate<T>>();
///results
private readonly Dictionary<string, T> Results = new Dictionary<string, T>();
private int _MaxParalelThreadCount;
private static Semaphore Semaphore;
public ThreadControler(int maxParalelThreadCount) {
MaxParalelThreadCount = maxParalelThreadCount;
}
public T GetResult(string key) {
if (!Results.ContainsKey(key))
throw new ArgumentException(string.Format("no result by this key {0}", key));
return Results[key];
}
public void AddDelegate(string key, FuncDelegate<T> @delegate) {
Delegates.Add(key, @delegate);
}
public int MaxParalelThreadCount {
get { return _MaxParalelThreadCount; }
set {
if (value < 1)
throw new ArgumentException("value<1");
_MaxParalelThreadCount = value;
}
}
///wrapper for delegate
public void ThreadFunc(object param) {
var threadParam = (ThreadParam)param;
var result = threadParam.Delegate();
lock (Results) {
Results.Add(threadParam.Id, result);
}
Semaphore.Release();
}
///run all delegate in parallel mode
public void Run() {
if (Delegates.Count == 0)
throw new InvalidOperationException("No Thread for Start");
if (MaxParalelThreadCount < 1)
throw new InvalidOperationException("MaxParalelThreadCount<1");
using(Semaphore = new Semaphore(MaxParalelThreadCount, MaxParalelThreadCount)){
var threads = new List<Thread>();
foreach (var key in Delegates.Keys) {
Semaphore.WaitOne();
var localKey = key;
var @delegate = Delegates[localKey];
var param = new ThreadParam { Delegate = @delegate, Id = localKey };
var thread = new Thread(ThreadFunc);
threads.Add(thread);
thread.Start(param);
}
foreach (var thread in threads)
thread.Join();
}
}
}
}
Web service for running and controlling application (TestComplete for example)
For Service creation I choose WCF. Here simple interface of service:
[ServiceContract]
public interface ISpreadRunnerService {
[OperationContract]
TaskResult TryRunProgramm(string @executable, string @params, int timeout);
}
For using this server need simple send “exe” name, params for “exe”, and timeout. Also we need to know busy server or not.
Service implementation:
[ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple)]
public class SpreadRunnerService : ISpreadRunnerService
{
private readonly object Locker = new object();
//strategy how to run app and how to get app results
protected IRunProgramStrategy Strategy;
public virtual TaskResult TryRunProgramm(string executable, string @params, int timeout)
{
//Lock service!!!
if (!Monitor.TryEnter(Locker))
{
return Strategy.NotRunResult;
}
var result= Strategy.Run(executable, @params, timeout);
Monitor.Exit(Locker);
return result;
}
/// <summary>
/// only for studio hoster
/// </summary>
internal SpreadRunnerService() : this(null) { }
public SpreadRunnerService(IRunProgramStrategy strategy)
{
Strategy = strategy;
}
}
SimpleRunStrategy code part. Only run application and return application exit code.
[DataContract]
public class TaskResult : ITaskResult {
[DataMember]
public bool Runned { get; set; }
[DataMember]
public bool Hang { get; set; }
[DataMember]
public int ProgramResult { get; set; }
[DataMember]
public string ExtraData;
}
public interface IRunProgramStrategy {
TaskResult Run(string executable, string @params, int timeout);
TaskResult HangResult { get; }
TaskResult NotRunResult { get; }
}
public class RunProgramStrategySimple : IRunProgramStrategy{
/*…*/
public virtual TaskResult Run(string executable, string @params, int timeout) {
TaskResult result=null;
try {
BeforeProcessRun();
Process p = new Process();
p.StartInfo.FileName = executable;
p.StartInfo.Arguments = @params;
p.Start();
p.WaitForExit(timeout);
if (!p.HasExited) {
p.Kill();
result= HangResult;
}
else
result= new TaskResult { Hang = false, ProgramResult = p.ExitCode, Runned = true };
}
catch (Exception e){
result= NotRunResult;
Console.WriteLine(e.Message + " "+executable+
" "+ @params);
}
finally{
AfterProcessRun(result);
}
return result;
}
/*…*/
Client is also very simple. Here it’s implementation:
public class Task
{
public string Id { get; set; }
public string Executable{get;set;}
public string Params{get;set;}
public int Timeout { get; set; }
}
public class SpreadRunnerClient
{
//our thread controller
private ThreadControler<TaskResult> ThreadController = new ThreadControler<TaskResult>();
//list of service host. May be better use UDDI but…
internal readonly List<string> Hosts = new List<string>();
//task –that stored in external file
private List<Task> Tasks = new List<Task>();
public IEnumerable<string> getHosts()
{
while(true)
foreach (string host in Hosts)
yield return host;
}
// create runner delegates for all task and run them in parallel mode
public void RunAllTasks()
{
ThreadController.MaxParalelThreadCount = Hosts.Count;
foreach (var task in Tasks)
{
var lockref = task;
FuncDelegate<TaskResult> func = delegate()
{
while (true)
{
foreach (string host in getHosts())
try
{
var binding = new WSHttpBinding("WSHttpBinding_ISpreadRunnerService");
binding.ReceiveTimeout = TimeSpan.FromMilliseconds(lockref.Timeout*2000);
binding.OpenTimeout= TimeSpan.FromMilliseconds(20000);
binding.SendTimeout = TimeSpan.FromMilliseconds(lockref.Timeout * 2000);
using (var client = new SpreadRunnerServiceClient(binding, new EndpointAddress(host)))
{
client.Open();
//try to run application in server
TaskResult result = client.TryRunProgramm(lockref.Executable, lockref.Params, lockref.Timeout);
if (result.Runned)
return result;
}
}
catch { }
}
throw new ApplicationException();
};
ThreadController.AddDelegate(lockref.Id,func);
}
ThreadController.Run();
}
And finally we need to make TestFixture that contain SpreadRunnerClient. In TestFixtureSetUp we must Load All Task from file and give them into SpreadRunnerClient. And call SpreadRunnerClient::RunAllTasks. For handling results we need to write some Test methods, for example:
private static void AssertTests(TaskResult res) {
bool isnormalresult = res.ProgramResult == 0 || res.ProgramResult == 1;
Assert.IsTrue(isnormalresult, "result is a " + res.ProgramResult + " |" + res.ExtraData);
Assert.IsFalse((res.ExtraData.ToLowerInvariant().Contains("exception")), res.ExtraData);
}
// ReSharper disable InconsistentNaming
[Test]
public void XXXX001() { var res = client.GetResultsById("XXXX001"); AssertTests(res); }
And finally we can run this test in TeamCity:
). RIA.NET code generator creates entity class on client side, where picture stored in field of type – byte[] (simple byte array).
