whitespace COMPANY whitespace SERVICES whitespace PRODUCTS whitespace PURCHASE whitespace SUPPORT whitespace CONTACTS whitespace Home whitespace Contact Us whitespace Site Map whitespace

UI Automation overview (Part II)


Project archive


For better understanding of this article you should have advanced knowledge of WPF, especially in the part of custom control creating.

In this part we will closely examine such fundamentals of UI Automation as:

  • UI Automation control patterns
  • UI Automation properties
  • UI Automation events

And in the end, we will try to implement a simple UI Automation support for custom control.

UI Automation control patterns

UI Automation control patterns provide a uniform approach to structure and express control's functionality. Microsoft identified the 18 control patterns, which represent everything that can be done with UI elements. For example, you can use Invoke control pattern for controls that can be invoked (such as buttons ) or Text pattern for edit controls and documents that expose textual information. There is no unique identification between patterns and controls, one pattern can belong to different controls as well as one control can support several patterns. For example: ComboBox supports ExpandCollapse and Selection patterns. So combining these 18 control patterns, developers can represent the full range of functionality for any user of UI element.

            Control Patterns and their Classes and Interfaces

Control Pattern Client-Side Class Provider-Side Interfaces
Dock DockPattern IDockProvider
ExpandCollapse ExpandCollapsePattern IExpandCollapseProvider
Grid GridPattern IGridProvider
GridItem GridItemPattern IGridItemProvider
Invoke InvokePattern IInvokeProvider
MultipleView MultipleViewPattern IMultipleViewProvider
RangeValue RangeValuePattern IRangeValueProvider
Scroll ScrollPattern IScrollProvider
ScrollItem ScrollItemPattern IScrollItemProvider
Selection SelectionPattern ISelectionProvider
SelectionItem SelectionItemPattern ISelectionItemProvider
Table TablePattern ITableProvider
TableItem TableItemPattern ITableItemProvider
Text TextPattern ITextProvider
Toggle TogglePattern IToggleProvider
Transform TransformPattern ITransformProvider
Value ValuePattern IValueProvider
Window WindowPattern IWindowProvider

Providers implement control pattern interfaces on UI elements. Control pattern interfaces are found in the System.Windows.Automation.Provider namespace and have names that include the suffix "Provider" (for example, IScrollProvider and IInvokeProvider ).
Clients access methods and properties of control pattern classes and use them to access information about a UI element, or to manipulate the UI. These control patterns classes are found in the Systems.Windows.Automation namespace and have names that include the suffix "Pattern" (for example, InvokePattern and SelectionPattern ).
Control patterns combine into one: structure, methods, properties and events supported by control. Control patterns relate to UI as interfaces relate to COM objects. In COM, you can query an object to ask what interfaces it supports, and then use those interfaces to access functionality. In UI Automation, clients can ask a control which patterns it supports and then interact with the control through the properties, methods, events, and structure of the supported control patterns. For example, providers implement IScrollProvider for a multi-line edit box. When a client detects that a UI element supports ScrollPattern, it can use the properties, methods, and events from that class to gather scroll-specific information or programmatically scroll its content to a new position.
Here is a small part of code that shows how to use InvokePattern
For example, we want to Invoke "Start" button. At first we need to find it and cast to AutomationElement, then to obtain its InvokePatterrn an call Invoke method:

AutomationElement start = null;   
PropertyCondition conds = new PropertyCondition( AutomationElement.NameProperty, "start" );    
start = AutomationElement.RootElement.FindFirst( TreeScope.Descendants, conds );   
if( start != null )   
   InvokePattern startInvoke = (InvokePattern)start.GetCurrentPattern( InvokePattern.Pattern );   

UI Automation properties

UI Automation properties are a set of standard properties that expose information, such as Name, ControlType, Orientation, AccessKey, etc., that is important to assistive technologies. These properties give an opportunity for UI Automation Client application to discover information about different parts of UI. UI Automation properties are read-only. To set properties of a control, you have to use the methods of the appropriate control pattern. Also, as you may notice from the previous code: properties (property ID's) are used in constructing PropertyCondition objects used to find AutomationElement objects. In that way you can specify (by Name, by Type, by Process Id, etc.) what kind of UI element you want to find.

The following code example shows two ways of retrieving a property on an AutomationElement.

// elementUI is an AutomationElement.   
// The following two calls are equivalent.   
string strName = elementUI.Current.HelpText;   
strName = elementUI.GetCurrentPropertyValue(AutomationElement.HelpTextProperty) as string;

To improve performance, property values of controls and control patterns can be cached when AutomationElement objects are retrieved. In UI Automation, caching means that the data can then be accessed without further cross-process communication. So you can get Current or Cached equivalent of property value or control pattern. For the most part, caching - is an automated process, but there are also special classes and methods that allow caching manipulations . The benefits of caching are most noticeable with WPF controls and custom controls that have server-side UI Automation providers. There is less benefit when accessing client-side providers such as the default providers for Win32 controls.
For different UI Frameworks the same UI Automation property can expose different native framework properties. The following table shows how one standard UI Automation property maps to multiple property names in other UI frameworks.

Mapping UI Automation Properties to Other UI Frameworks
UI Automation Control Type UI Framework Framework Property UI Automation Property
Button Avalon Content NameProperty
Button Win32 Caption NameProperty
Image HTML ALT NameProperty

By implementing UI Automation, providers should map unique UI framework properties to standard UI Automation properties.

UI Automation events

UI Automation provides information to client applications through events. Unlike WinEvents, UI Automation events are not based on a broadcast mechanism. Clients register for specific event notifications and can request that specific UI Automation properties and control pattern information be passed into their event handler. Providers can improve performance by raising events selectively, depending on whether any clients are subscribed to those events, or not.

UI Automation events fall into the following categories
Event Description
Property change Raised when a property on an UI Automation element or control pattern changes. For example, if a client needs to monitor an application's check box control, it can register to listen for a property change event on the ToggleState property. When the check box control is checked or unchecked, the provider raises the event and the client can act as necessary.
Element action Raised when a change in the UI results from end user or programmatic activity; for example, when a button is clicked or invoked through InvokePattern
Structure change Raised when the structure of the UI Automation tree changes. The structure changes when new UI items become visible, hidden, or removed on the desktop.
Global desktop change Raised when actions of global interest to the client occur, such as when the focus shifts from one element to another, or when a window closes.

Here is an example on how to track Start button Invoke event:

// start is AutomationElement.that represents Start button   
//Register an event handler for InvokedEvent on the start button.   
Automation.AddAutomationEventHandler(InvokePattern.InvokedEvent, start,
   TreeScope.Element, new AutomationEventHandler(OnStartInvoke));   
//Event handler   
private void OnStartInvoke( object src, AutomationEventArgs e )   
   MessageBox.Show( "Start has been invoked" );   

Implementing UI Automation support

This is a small tutorial about how to provide an UI Automation support for custom WPF control. To demonstrate this, I have created two custom buttons (SimpleButton and SimpleButton2) with 3D effect click. This two controls are identional, but one of them supports UI Automation(SimpleButton) and another(SimpleButton2) does not. I've done it, to show that control with UI Automation support can be recognized by accessibility programs, and another one that does not provide UI Automation support can not.

Note: To feel all details of implementing UI Automation support, it is very important that control should be derived from low - hierarchy class (like Control or ContentControl) and would not have a standard WPF control inside its template. Because standard WPF controls support UI Automation by default.

Microsoft gives a special requirements (such as tree structure, properties, supported patterns, etc ) necessary for correct UI Automation implementation. You can find them here

Custom WPF controls provide UI Automation support through AutomationPeer class or derived from it. There are also many specialized classes such as: ButtonAutomationPeer CheckBoxAutomationPeer, etc. But as our SimpleButton derives from ContentControl class our peer class will be derived from FrameworkElementAutomationPeer. The good rule is to name it SimpleButtonAutomationPeer (control name + "AutomationPeer").
Here is an implementation of SimpleButtonAutomationPeer:

#region UI Automation support   
    /// <summary>   
    /// Class that provides UI Automation support   
    /// </summary>   
    public class SimpleButtonAutomationPeer : FrameworkElementAutomationPeer, IInvokeProvider   
        public SimpleButtonAutomationPeer( SimpleButton control )   
            : base( control )   
        protected override string GetClassNameCore()   
            return "SimpleButton";   
        protected override string GetLocalizedControlTypeCore()   
            return "button";   
        protected override AutomationControlType GetAutomationControlTypeCore()   
            return AutomationControlType.Button;   
        public override object GetPattern( PatternInterface patternInterface )   
            if( patternInterface == PatternInterface.Invoke )   
                return this;   
            return base.GetPattern( patternInterface );   
        private SimpleButton MyOwner   
                return (SimpleButton)base.Owner;   
        #region IInvokeProvider Members   
        public void Invoke()   
            RoutedEventArgs newEventArgs = new RoutedEventArgs( SimpleButton.ClickEvent );   
            MyOwner.RaiseEvent( newEventArgs );   

In this class we overrode some basic property getters which are responsible for correct control identification. As our control is a button we also should support InvokePattern and or TogglePattern (see requirements) To do this, IinvokeProvider interface must be implemented. If the control supports the Invoke control pattern, then InvokeEvent must be supported too. Therefore before raising Click RoutedEvent, we should raise UI Automation Invoke event(see code below).

protected virtual void OnClick()
   //raising UI Automation InvokeEvent
   if( AutomationPeer.ListenerExists( AutomationEvents.InvokePatternOnInvoked ) )
        AutomationPeer peer = UIElementAutomationPeer.CreatePeerForElement( this );
        if( peer != null )
            peer.RaiseAutomationEvent( AutomationEvents.InvokePatternOnInvoked );
   //raising routed event click

And the last we should do is to override OnCreateAutomationPeer method in SimpleButton class, which returns SimpleButtonAutomationPeer object( see code below ).

protected override AutomationPeer OnCreateAutomationPeer()   
   return new SimpleButtonAutomationPeer( this );   

To see the results of all this work I have created an application that contains several controls, among which there are standard WPF controls and two our custom buttons(one with UI Automation support, and another one - without ) . I also added a simple UI Automation explorer that builds a tree and gives some information about controls with UI Automation support, which belong to its own application.

Don't forget! If you want to explore your own application user interface, you must make all UI Automation calls from a separate thread.

As you can notice at the screenshot, MySimpleButton has an UI Automation support, therefore you can find it in the UI tree, and MySimpleButton2 (without UI Automation support) could not be detected by UI Automation system. You can also check it with UI Spy utility.

It's a good idea to run UI Spy against your application to make sure everything is shown correctly. If an item appears incorrectly, you can manually set the property which provides a wrong information. You can do this in such way, for example HelpText property is incorrect:

<TextBlock Name="textBox" AutomationProperties.HelpText="Correct help text"/>

So, no matter how good your control is, if you want to make it accessible for people with special needs, you must support UI Automation. As it was shown at the latest surveys the number of such people increases, therefore supporting UI Automation will improve product quality and will be a good thing in attracting clients.

< Previous | Home

Author: 2007 Dima Zaharov, Department Manager/ArtfulBits
Company | Services | Practices | Technologies | Career | Contacts | Privacy
© 2005-2019 ArtfulBits. All rights reserved.