Hybrid Test Automation Framework

Hybrid Test Automation

The most commonly implemented framework is a best combination of all the techniques. It combines Keyword Driven, modular, Library and Data Driven frameworks. Hybrid Testing Framework allows data driven scripts to take advantage of the powerful libraries and utilities that usually accompany a keyword driven architecture. The framework utilities can make the data driven scripts more compact and less prone to failure. Tests are fully scripted in a Hybrid Testing Framework thus increasing the automation effort. Hybrid Testing Framework also implements extensive error and unexpected windows handling. It is used for automation of medium to large applications with long shelf life. 

Advantages:

§   Fastest and less costly way to develop the automation scripts due to higher code re- usability

§    Utilizing a modular design, and using files or records to both input and verify data, reduces redundancy and duplication of effort in creating automated test scripts

 

Disadvantages:

§    No reporting mechanism to maintain reports for test results, error logs

§    No separate function libraries for common code and business logic code

Data-Driven Framework 

Data-driven testing is a framework where test input and output values are read from data files (data pools, ODBC sources, CSV files, Excel files, DAO objects, ADO objects etc.) and are loaded into variables in captured or manually coded scripts. In this framework, variables are used for both input values and output verification values. Navigation through the program, reading of the data files, and logging of test status information are all coded in the test script.

This is similar to table-driven testing where test case is contained in the data file and not in the script; the script is just a "driver," or delivery mechanism, for the data. Unlike in table-driven testing, though, the navigation data isn't contained in the table structure, In data-driven testing, only test data is contained in the data files.

                  

Advantages:

§    Data sheets can be designed while application development is still in progress

§    Reduces data redundancy

§    Data input/output and expected results are stored as easily maintainable text records in    database

§    Changes to the Test Scripts do not affect the Test Data

§    Test Cases can be executed with multiple Sets of Data

Disadvantages:

§    Requires database proficiency

§ Maintenance of database is required

Library Framework

Test Library Architecture

The test library architecture framework is very similar to the test script modularity framework and offers the same advantages, but it divides the application-under-test into procedures and functions instead of scripts. This framework requires the creation of library files which has reusable code represented in modules, sections and functions of the application-under-test. These library files are then called directly from the test case script.

           

Advantages:

§     Performance of automation scripts increases due to higher code re-usability

§    Functionality changes can be accommodated easily by updating only the specific "Business Function" script

§   Object recognition changes can be easily accommodated in tools without inbuilt object repository

Disadvantages:

§    Technical expertise is necessary to write Scripts using Test Library Framework

§    More time is needed to plan and prepare test scripts

§    Cannot drive the data from database

Modular Framework

Test Script Modularity

The test script modularity framework requires the creation of small, independent scripts that represent modules, sections, and functions of the application-under-test. These small scripts are then used in a hierarchical fashion to construct larger tests, realizing a particular test case.

Of all the frameworks, this is the simplest to grasp and master. It's a well-known programming strategy to build an abstraction layer in front of a component to hide the component from the rest of the application. This insulates the application from modifications in the component and provides modularity in the application design. The test script modularity framework applies this principle of abstraction or encapsulation in order to improve the maintainability and scalability of automated test suites.

Advantages:

§    Fastest way to generate script

§    Modular division of scripts leads to easier maintenance

Disadvantages:

§   Cannot make use of reusable libraries

§   Cannot drive the data from database

Keyword Driven Framework

Keyword-Driven or Table-Driven Testing

Keyword-driven testing and table-driven testing are interchangeable terms that refer to an application-independent automation framework. This framework requires the development of data tables and keywords which are independent of the test automation tool used to execute them, and the test script code that "drives" the application-under-test and the data. Keyword-driven tests look very similar to manual test cases. In a keyword-driven test, the functionality of the application-under-test is documented in a table as well as in step-by-step instructions for each test.

There are 2 basis components in Keyword Driven Framework viz. Keyword, Application Map.

What is a Keyword or Action?

Keyword is an Action that can be performed on a GUI Component. Ex. For GUI Component Textbox, some Keywords (Action) would be InputText, VerifyValue, and VerifyProperty and so on. 

What is Application Map or Control?

An Application Map provides Named References for GUI Components. Application Maps are nothing but ‘Object Repository’. 

Example:  The following table represents Keyword driven framework for ‘Calculator’ application. The "Window" column contains the name of the application window where we're performing the mouse action (in this case, they all happen to be in the Calculator window). The "Control" column names the type of control the mouse is clicking. The "Action" column lists the actions taken with the mouse (or by the tester). And the "Arguments" column names a specific control (1, 2, 3, 5, +, -, and so on).

Window	Control	Action	Arguments
Calculator	Menu		View, Standard
Calculator	Pushbutton	Click	1
Calculator	Pushbutton	Click	+
Calculator	Pushbutton	Click	3
Calculator	Pushbutton	Click	=
Calculator		Verify Result	4
Calculator		Clear
Calculator	Pushbutton	Click	6
Calculator	Pushbutton	Click	-
Calculator	Pushbutton	Click	3
Calculator	Pushbutton	Click	=
Calculator	*	Verify Result	3

Advantages:

§    Detail Test Plan can be written in Spreadsheet format containing all input and  verification data.

§    Prior to the Detail Test Plan being written, if "utility" scripts are created by people who are well versed with the automated tool’s and Scripting language, then the tester can use the Automated Test Tool immediately via the "spreadsheet-input" method, without needing to learn the Scripting language.

§    The tester needs to learn only the "Key Words" required, and the specific format to use within the Test Plan. This allows the tester to be productive with the test tool very quickly, and allows more extensive training in the test tool to be scheduled at a more convenient time.

Disadvantages:

§    Development of "customized" (Application-Specific) Functions and Utilities requires proficiency in the tool’s Scripting language.

§    If application requires more than a few "customized" Utilities, the tester has to learn a number of "Key Words" and special formats. This can be time-consuming, and may have an initial impact on Test Plan Development. Once the testers get used to this, however, the time required to produce a test case will be greatly improved.

Automation Framework

A Test Automation Framework is a set of assumptions, concepts and tools that provide support for automated software testing. It works like a wrapper between the application and the tool. It will reduce the programming effort and maintenance effort. The Test automation framework must be evolving continuously to handle new complexities in applications.

Need of automation framework

Implementation of appropriate automated testing framework helps the enterprise to significantly increase the speed and accuracy of the testing process, provides a higher return on investment from software project and systematically minimizes risk. The framework should be such that maintainability of scripts is easy and the scripts created are robust.

            When scripts are maintained without a planned procedure, maintenance becomes difficult when AUT (application under test) is updated with new changes.

Benefits of robust automation framework

§  Faster test scripts generation

§  Data driven test

§  Coded scripts are enhanced with the design

§  Increases code re-usage

§  Higher portability

§  Ease of maintenance

Types of Framework

There are several test automation frameworks available, among these the selection is made based on the factors such as re-usability of scripts and test assets. Different test automation frameworks available are:

v  Keyword-Driven or Table-Driven Testing

v  Test Script Modularity

v  Test Library Architecture

v  Data-Driven Testing

v  Hybrid Test Automation

508 Compliance Testing

What is '508 Compliance' testing?

An application is said to be ‘508 Compliant’ if it provides access support for individuals with disabilities. The purpose of this part is to implement section 508 of the Rehabilitation Act of 1973, as amended (29 U.S.C. 794d).

Rules for '508 Compliance'

There are 16 rules for 508 Compliance. A tester has to identify which component/ functionality is not compliant with a rule.

a). A text equivalent for every non-text element shall be provided (e.g., via "alt", "longdesc", or in element content).

b). Equivalent alternatives for any multimedia presentation shall be synchronized with the presentation

c). Web pages shall be designed so that all information conveyed with color is also available without color, for example from context or markup

d). Documents shall be organized so they are readable without requiring an associated style sheet

e). Redundant text links shall be provided for each active region of a server-side image map

f). Client-side image maps shall be provided instead of server-side image maps except where the regions cannot be defined with an available geometric shape

g). Row and column headers shall be identified for data tables.

h). Markup shall be used to associate data cells and header cells for data tables that have two or more logical levels of row or column headers

i). Frames shall be titled with text that facilitates frame identification and navigation.

j). Pages shall be designed to avoid causing the screen to flicker with a frequency greater than 2 Hz and lower than 55 Hz.

k). A text-only page, with equivalent information or functionality, shall be provided to make a web site comply with the provisions of these standards, when compliance cannot be accomplished in any other way. The content of the text-only page shall be updated whenever the primary page changes

l). When pages utilize scripting languages to display content, or to create interface elements, the information provided by the script shall be identified with functional text that can be read by assistive technology

m). When a web page requires that an applet, plug-in or other application be present on the client system to interpret page content, the page must provide a link to a plug-in or applet

n). When electronic forms are designed to be completed on-line, the form shall allow people using assistive technology to access the information, field elements, and functionality required for completion and submission of the form, including all directions and cues

o). A method shall be provided that permits users to skip repetitive navigation links

p). When a timed response is required, the user shall be alerted and given sufficient time to indicate more time is required

Free toolbars for '508 Compliance' testing

§ 508 Web accessibility toolbars: http://jimthatcher.com/webcourse3.htm

§ Cynthia Says: http://www.cynthiasays.com/

§  Web Accessibility Checker: http://achecker.ca/checker/index.php

§  Firefox Accessibility Evaluation Toolbar https://addons.mozilla.org/en-US/firefox/addon/accessibility-evaluation-toolb/

§  IE accessibility toolbar - http://www.visionaustralia.org.au/ais/toolbar/

§  Firefox - https://addons.mozilla.org/en-US/firefox/addon/juicy-studio-accessibility-too/ and https://addons.mozilla.org/en-US/firefox/addon/web-developer/