Windows® 7 offers end-to-end accessibility with better performance, seamless interoperability, and improved framework design.

The Windows Automation API 3.0 combines the advantages of existing accessibility implementations in Microsoft® Active Accessibility® and a modern API design in UI Automation to provide easier migration from and integration with precursor technologies and industry standards.

The major improvements in this accessibility framework are:

• End-to-end unmanaged solution, including a native client API, faster and more robust UI Automation implementations, extensible provider implementations, and Win32 proxy for performance and availability.
• Harmonization with industry standards such as W3C ARIA and Section 508 specifications.
• New properties and control patterns.
• Custom properties, events, and patterns.

An Unmanaged Client API

For Windows 7, UI Automation has a new COM client API for use from unmanaged and managed clients alike. Unmanaged applications can now use UI Automation without changing languages or loading the CLR, while simultaneously benefiting from all the latest features. This new API is similar to the previous managed API, but is friendlier to C++ developers.

Unmanaged applications can now use UI Automation without changing languages or loading the CLR, while simultaneously benefiting from all the latest features.

The heart of the new COM API is the IUIAutomation interface, which enables clients to get automation elements, register event handlers, create objects, and access other helper methods. To begin using UI Automation, simply include the Automation header (UIAutomation.h), and then CoCreate the Automation object:

IUIAutomation * pAutomation;
CoCreateInstance(__uuidof(CUIAutomation), NULL,
                 CLSCTX_INPROC_SERVER,
                 __uuidof(IUIAutomation),
                 (void **)&pAutomation);

Once you have the Automation object, you can discover the entire user interface (UI). The UI is modeled as a tree of automation elements (IUIAutomationElement objects), each element representing single pieces of UI: a button, a window, the desktop, and so on. The IUIAutomationElement interface has methods relevant to all controls, such as checking properties or setting focus. Here, you get the element with focus and its name:

IUIAutomationElement *pElement;
pAutomation->GetFocusedElement(&pElement);
BSTR name;
pElement->get_CurrentName(&name);
std::wcout << L"Focused Name:" << name << L"\n";

Furthermore, elements can expose functionality specific to a particular control through control patterns. Control patterns are collections of associated properties, events, and methods, and more than one can apply to an element. Here, I use the Invoke pattern to press a button:

IUIAutomationInvokePattern * pInvoke;
pElement->GetCurrentPatternAs(UIA_InvokePatternId,
             __uuidof(IUIAutomationInvokePattern),
             (void **)&pInvoke);
pInvoke->Invoke();

To navigate the tree of automation elements, you can use tree walkers. Here, I create an IUIAutomationTreeWalker object to navigate to the first child of an element in the Control View of the tree:

IUIAutomationTreeWalker * pWalk;  
pAutomation->get_ControlViewWalker(&pWalk);
IUIAutomationElement * pFirst;
pWalk->GetFirstChildElement(pElement, &pFirst);

To address performance issues when communicating across processes, clients can fetch multiple properties and patterns at a time. Here, I create a cache request (IUIAutomationCacheRequest object) from the Automation object, identify the property (Automation ID property) to prefetch, cache it for an element, and then get the cached ID:

IUIAutomationCacheRequest * pCR;
pAutomation->CreateCacheRequest(&pCR));
pCR->AddProperty(UIA_AutomationIdPropertyId);

IUIAutomationElement * pCachedElement;
pElement->BuildUpdatedCache(&pCachedElement);

BSTR autoID;
pCachedElement->get_CachedAutomationId(&autoID);
std::wcout << L"Cached ID:" << autoID << L"\n";

With UI Automation property conditions, looking for specific values for a property is easy; you can combine conditions with AND, OR, and NOT operators to search for UI scenarios. Here, I search for a check box:

IUIAutomationCondition * pCheckBoxProp;
VARIANT varCheckBox;
varCheckBox.vt = VT_I4;
varCheckBox.lVal = UIA_CheckBoxControlTypeId;
pAutomation->CreatePropertyCondition(
    UIA_ControlTypePropertyId,
    varCheckBox, &pCheckBoxProp);

IUIAutomationElement * pFound;
pElement->FindFirst(TreeScope_Descendants,
                    pCheckBoxProp, &pFound);

In addition to these features, UI Automation provides custom proxy registration, events, and the Text Pattern for manipulating documents.

Proxy Factory

Clients can customize how UI Automation sees HWND-based controls by registering customized providers with a proxy factory implementation. Custom proxies affect only the client’s own view. The provider answers CreateProvider calls with the HWND to be proxied, as well as the idObject and idChild associated with the WinEvent, if needed.

Here, I verify the expected values of OBJID_CLIENT and CHILDID_SELF and create a custom proxy:

IFACEMETHODIMP CustomFactory::CreateProvider(
    UIA_HWND hwnd, LONG idObject, LONG idChild,
    IRawElementProviderSimple **ppRetVal)
{
    *ppRetVal = NULL;
    if(idObject == OBJID_CLIENT &&
       idChild  == CHILDID_SELF)
    {
        // Create the custom proxy.
        *ppRetVal = new CustomProxy((HWND)hwnd);
    }
    return S_OK;
}

Proxy factory registration consists of creating proxy factory entries and inserting them into a proxy factory mapping. The mappings are processed in order as clients register proxies. In this example, I create the proxy for all HWNDs with a class name containing the word “MYCUSTOMBUTTON”:

// Instantiate the proxy factory.
IUIAutomationProxyFactory * pCF =
    new CustomFactory();

// Create an entry with the factory.
IUIAutomationProxyFactoryEntry * pCEnt;
pAutomation->CreateProxyFactoryEntry(pCF, &pCEnt);
pCEnt ->put_ClassName(L"MYCUSTOMBUTTON");
pCEnt ->put_AllowSubstringMatch(TRUE);

// Get the mapping.
IUIAutomationProxyFactoryMapping * pMapping;
pAutomation->get_ProxyFactoryMapping(&pMapping);

// Insert the entry at the start of the mapping.
pMapping->InsertEntry(0, pCEnt);

Optimizations in the Event Model

UI Automation clients, like screen readers, track events raised in the UI by UI Automation providers and notify users of the events. To improve efficiency in Windows 7, providers can raise an event selectively, notifying only those clients that subscribe to that event.

Common UI Automation events that providers should support include the following:

• Changes in a property or control pattern for a UI Automation element.
• End user or programmatic activity that affects the UI.
• Changes to the structure of the UI Automation tree.
• Shift in focus from one element to another.

To listen to an event, create a COM object implementing the event handler interface, and then call the corresponding method on the IUIAutomation object to subcribe the handler to the event. Here, I subscribe a handler to the Focus Changed event:

class FocusChangedHandler:
    public IUIAutomationFocusChangedEventHandler;

FocusChangedHandler * focusHandler =
    new FocusChangedHandler();

// Register the event with default cache request.
pAutomation->AddFocusChangedEventHandler(
    NULL, focusHandler);

Interoperability with Microsoft Active Accessibility-based Controls

To improve interoperability, UI Automation translates between Microsoft Active Accessibility and UI Automation implementations. UI Automation clients can use the new UI Automation services to interact with earlier Microsoft Active Accessibility implementations, and Microsoft Active Accessibility clients can interact with UI Automation providers.

Clients using the UI Automation client API can use its services to search existing IAccessible implementations. Code that writes to the UI automation provider interfaces will still be visible to existing IAccessible clients.

In Windows 7, Microsoft Active Accessibility implementations can add UI Automation properties and control patterns by supporting the IRawElementProviderSimple interface to expose patterns and properties and the IAccessibleEx interface to handle ChildIds. Figure 1 shows the relationship between the IAccessible, IAccessibleEx, IRawElementProviderSimple, and IRangeValueProvider interfaces.

Figure 1: COM diagram shows the role of IAccessibleEx in extending legacy implementations.

With the IAccessibleEx interface, developers can extend Microsoft Active Accessibility implementations by adding required UI Automation object model information. The new MSAA-to-UIA proxy featured in the Windows Automation API provides a variation of a UI Automation “provider” to Microsoft Active Accessibility implementations. UI Automation clients can interact with all variations of UI Automation implementations: native UI Automation, IAccessible (Microsoft Active Accessibility), and IAccessible + IAccessibleEx.

New Win32 Support via OLEACC Plus IAccessibleEx

In Windows 7, OLEACC proxies expose information about common controls that Microsoft Active Accessibility cannot express. The MSAA-to-UIA proxy recognizes IAccessibleEx and forwards this additional information to UI Automation. For example, the OLEACC slider proxy adds the IRangeValue pattern, exposing minimum and maximum values that Microsoft Active Accessibility cannot expose. Extending the OLEACC proxies with IAccessibleEx has the dual benefit of leveraging existing code and keeping the OLEACC proxies up to date.

Some frameworks such as HTML, Windows Presentation Foundation (WPF), and Silverlight™ have a metadata system that can associate accessibility-related properties with an element. Consequently, developers can easily fix common bugs such as an incorrect accessibility name. However, Win32 does not have a similar feature, and the very basic property system for HWNDs does not apply to sub-items.

However, with Direct Annotation, developers can mark up properties on Win32 controls with accessibility property/value information, enabling developers to fix bugs without needing a full UI Automation or Microsoft Active Accessibility implementation. Here, I set the AutomationId on a simple control with the hwnd:

IAccPropServices *pAccPropServices;
CoCreateInstance(CLSID_AccPropServices, NULL,
    CLSCTX_SERVER, IID_IAccPropServices,
    (void**)&pAccPropServices);
pAccPropServices->SetHwndPropStr(hwnd,
    OBJID_CLIENT, CHILDID_SELF,
    AutomationId_Property_GUID, L"Foo ID");

Custom Control Patterns, Properties, and Events

With Windows 7, you can extend the platform with custom control patterns, properties, and events. Because of this support, developers of UI Automation clients and providers can introduce new accessibility specifications independent of future operating system releases.

Developers register and use custom patterns, properties, and events on both the client side and the provider side. If a provider registers for a property that the client hasn’t, the client can’t retrieve it, and if a client registers for a property that the provider hasn’t, the provider can’t respond to a request for it. Neither of these cases causes errors; one side merely remains unaware of the other’s capabilities.

Once the new property, pattern, or event is registered, using it is just like using a built-in pattern, property, or event.

Custom Properties

Registration of properties is identical for both client and provider. In the following sample, I have a native UI Automation control and I want to add a custom string property, LongName:

// The PropertyId for the LongName property.
PROPERTYID longNamePropertyId;

// This is the predefined property GUID,
// the name of the property, and the type.
UIAutomationPropertyInfo longNamePropertyInfo = {
    GUID_LongNameProp,
    L"LongName",
    UIAutomationType_String };

// This yields the property ID for new property.
pAutomationRegistrar->RegisterProperty(
    &longNamePropertyInfo, &longNamePropertyId);

Retrieving the property is similar to retrieving a normal property; you use the property ID that the RegisterProperty method initialized:

VARIANT longNameValue;
pElement->GetCurrentPropertyValue(
    longNamePropertyId, &longNameValue);
std::wcout << longNameValue.bstrVal << L"\n";

Again, on the provider side, you call the same registration method except you add an entry in the control’s implementation of IRawElementProviderSimple::GetPropertyValue.

In this code sample, the UI Automation implementation keeps a reference to the control in _pOurControl, and the control supports a GetLongName method:

GetPropertyValue(PROPERTYID propertyId,
                 VARIANT * pRet)
{
    pRet->vt = VT_EMPTY;

    if(propertyId == longNamePropertyId)
    {
        // Get the long name from the control.
        pRet->bstrVal = _pControl.GetLongName();
        pRet->vt = VT_BSTR;
    }

    // Deal with other properties...
    return S_OK
}

Custom Events

Events follow a nearly identical model to properties: you register the event ID, and then you can use the custom EventID exactly as you use a normal EventID. However, custom events cannot have arguments; they are merely notifications that the event occurred.

Here, I register the event. Both listening and raising would then be identical to other events:

// The EventId for the ThingHappened event
EVENTID thingHappenedEventId;

// Event Information
// This is the event GUID, the name of the event.
UIAutomationEventInfo thingEventInfo = {
    GUID_ThingHappenedEvent, L"ThingHappened" };

// This gives you the Event ID for the new event.
pAutomationRegistrar->RegisterEvent(
    &thingEventInfo, &thingHappenedEventId);

Custom Control Patterns

Creating a custom control pattern requires the following:

• Arrays of events, properties, and methods associated with the pattern.
• IIDs of the pattern’s corresponding provider interface and client interface.
• Code to create a client interface object.
• Code to perform marshalling for the pattern’s properties and methods.

On the client side, the code that registers a pattern must supply a factory for creating instances of a Client Wrapper that forwards property requests and method calls to an IUIAutomationPatternInstance provided by UI Automation. The UI Automation framework then takes care of remoting and marshalling the call.

Windows 7 offers further extensibility through the registration of custom control patterns, properties, and events.

On the provider side, the code that registers a pattern must supply a “pattern handler” object that performs the reverse function of the Client Wrapper. The UI Automation framework forwards the property and method requests to the pattern handler object, which in turn calls the appropriate method on the target object’s provider interface.

The UI Automation framework takes care of all communication between the client and provider, both of which register corresponding control pattern interfaces. For more details please refer to the Windows 7 SDK.

New Properties and Control Patterns

Some other useful properties for automation elements have been added to UI Automation for Windows 7:

However, with Direct Annotation, developers can mark up properties on Win32 controls with accessibility property/value information…

ControllerFor is an array of elements manipulated by the automation element. Without this property it is hard to determine the impact of an element’s operation.

DescribedBy is an array of elements that provide more information about the automation element. Instead of using the object model to discover information about the element, clients can quickly access that information in the DescribedBy property.

FlowsTo is an array of elements that suggest the reading order after the current automation element. FlowsTo is used when automation elements are not exposed or structured in the reading order users perceive.

IsDataValidForForm identifies whether data is valid in a form.

ProviderDescription identifies source information for the automation element’s UI Automation provider, including proxy information.

Control Patterns for Virtualized Child Objects

When a control has too many children to load at once, a common solution is to treat the excess children as virtualized controls. This creates problems for the UI Automation tree map because there are only a handful of real controls and the virtualized controls simply don’t exist in the UI Automation tree.

To manage this, UI Automation offers two control patterns. The ItemContainer pattern lets a user search a container of virtualized controls for specific properties. This gives the client a reference to a virtualized control, but the user can’t do anything with it. The VirtualizedItem pattern enables the client to force the item to exist, either by realizing it internally, or by having it scroll on screen.

In this client-side code example, I search for a specifically named item in a virtualized list:

// Get the ItemContainer pattern.
IUIAutomationItemContainerPattern * pContainer;
pElement->GetCurrentPatternAs(
  UIA_ItemContainerPatternId,
  __uuidof(IUIAutomationItemContainerPattern),
  (void**)&pContainer));

// Search the container for the property.
VARIANT varNameStr;
varNameStr.vt = VT_BSTR;
varNameStr.bstrVal = SysAllocString(name);

IUIAutomationElement * pFound;
pContainer->FindItemByProperty(NULL,
  UIA_NamePropertyId, varNameStr, &pFound);

// Realize the virtual element.
IUIAutomationVirtualizedItemPattern * pVirt;
pFoundElement->GetCurrentPatternAs(
  UIA_VirtualizedItemPatternId,
  __uuidof(IUIAutomationVirtualizedItemPattern),
  (void**)&pVirt));

pVirtualizedItem->Realize();

New UI Automation Properties for Accessible Rich Internet Applications (ARIA)

Every day, Web sites are increasing their utility with dynamic content and advanced UI controls by using technologies like Asynchronous JavaScript and XML (AJAX), HTML, and JavaScript. However, assistive technologies are frequently unable to interact with these complex controls or expose dynamic content to users. Accessible Rich Internet Applications (ARIA) is a W3C technical specification for developing Web content and applications so that they are accessible to people with disabilities.

To support the ARIA specification, the UI Automation specification enables developers to associate UI Automation AriaRole and AriaProperties attributes with W3C ARIA Roles, States, or Properties. This helps user applications such as Internet Explorer support the ARIA object model in the context of UI Automation while keeping a baseline accessibility object model.

Some parts of the ARIA specification can be mapped to the desktop-oriented Microsoft Active Accessibility object model; however, much of the specification can only be applied to rich internet applications. Table 1 lists some examples of mappings from W3C ARIA Roles to Microsoft Active Accessibility Roles and UI Automation Control Types.

For example, the ARIA Role checkbox is supported in Microsoft Active Accessibility by the role ROLE_SYSTEM_CHECKBUTTON and in UI Automation by the combination of control type Checkbox and AriaRole checkbox. The ARIA state checked is supported in Microsoft Active Accessibility by the state STATE_SYSTEM_CHECKED and in UI Automation by the control pattern Toggle Pattern and the AriaProperties property checked.

ARIA States and Properties are supported by the UI Automation AriaProperties property with the following exceptions: ARIA properties that take object references (like the describedby property), and ARIA properties already supported by the accessibility object model. Table 2 lists examples of mappings from W3C ARIA States and Properties to various properties and functions of Microsoft Active Accessibility and UI Automation.

To improve interoperability, UI Automation translates between Microsoft Active Accessibility and UI Automation implementations.

UI Automation also offers a simple text object model with the TextPattern pattern, which supports embedding objects in a document object. This enables user agents and client applications to treat Web content either as an HTML document or as a traditional desktop UI depending on the end-user scenarios.

With these features, UI Automation enables both the support of and extension of the W3C ARIA specification without a dependency on a specific application or browser.

Conclusion

With application user interfaces growing more and more complex, getting accessibility right is a challenge for developers. Programmatic access to the UI is critical in the development of assistive technologies like screen readers and magnifiers. To address this, the Windows 7 Automation API aims to provide a complete end-to-end, flexible, extensible, and consistent framework with improved design and performance.

For further details, please refer to the Windows 7 SDK or the MSDN Accessibility Developer Center (http://msdn.microsoft.com/accessibility/).