In our industry, the only constant is change itself, and application development methodologies and techniques are constantly in flux. However, even for our industry, things seem to have changed at a faster than usual pace over the recent years. Modern applications follow a distinctly different pattern than applications written only two or three years ago. In the past, application performance was of utmost importance. Today, security and scalability are the ruling factors. Where in the past, a single EXE was seen as "the application", today, applications are distributed and offer any number of interfaces to them. Users have much higher expectations in how their applications are updated and maintained. Many even demand applications that can work entirely without a client-side component to avoid a cumbersome installation procedure. And of course, applications need to work across the Internet, no matter whether they are Windows applications or browser-based.
Coping with these and similar demands can be difficult for developers. The industry has moved from Windows applications to HTML applications to provide appropriate availability and reach. The advent of web services is now swinging the pendulum the other way, providing browser-like connectivity in Windows applications, with the added benefit of full Windows UI power and offline capabilities. On top of that, there is a great demand for mobile device integration, such as Pocket PCs and Smart Phones.
However, user demand is not the only driving force. Application and Enterprise Integration is growing more and more important, calling for mechanisms such as XML Web Services and Service Oriented Architecture (SOA).
Security and trustworthiness represent a huge challenge. Secure architectural components such as SQL Server are part of the story, but security is a "weakest link" problem. Adding secure components does not help very much as long as non-secure components exist in a system.
In an attempt to come up with answers for all these demands, application architecture and involved technologies have undergone many drastic changes. The most important ones I will highlight in this paper. Before we look at many of these aspects, it is important to understand how Visual FoxPro has evolved over the last decade (or more) to cope with new scenarios, and how that relates to today's demands.
A Short History of Visual FoxPro Development
Visual FoxPro has been around for a while (especially if you count earlier versions such as FoxPro and FoxBase). It is a mature product that is very stable. As a VFP developer, you are very familiar with the product, and also very productive in its development environment. So why would you ever move away and use a different language, development environment, or database back end?
I can suggest many answers to this question. For one, perhaps there is no good reason to move. Maybe whatever you are currently doing is perfectly fine for the scenarios you are implementing. Or perhaps you do not want to move, but instead add to your toolbox. Visual FoxPro is extremely good at what it does, but it also focuses on a specific subset of the development world. Expanding your horizons a bit is never a bad idea.
A good number of people tell me that they are interested in going a bit beyond that though. The computer industry is changing at a very rapid pace, making it unrealistic to expect that "everything you need to do, you can do in Visual FoxPro today", as some old-time "gurus" will try to make you believe. In fact, considering that Visual FoxPro (even in its latest version) is based on ideas dreamt up more than a decade ago, it is not surprising that it can not fulfill all the needs of modern business scenarios.
The "Bad Old Days" of Building Applications
For Visual FoxPro developers, the application development landscape seemingly has changed drastically over the last 15 years. But is that really so? Modern VFP software development has moved from a procedural environment into an object-oriented world (and by now, VFP developers arguably know objects better than most business developers). This was a rather drastic paradigm shift in application building. In fact, the paradigm shift was so great, that it was hardly possible to move FoxPro 2.x-based applications into the "Visual" world. (A fact many companies are still struggling with today).
Looking back at the change today, the shift was large, yet at the end of the day, developers simply found a better way to build the same sort of application that FoxPro had been building for years. What changed? Developers could do more of the same much easier, more manageable, and at a higher level of quality. However, developers still built "monolithic" applications that at best used a "client/server" architecture. In fact, one could argue that even the oldest FoxBase application basically had two distinct layers: The logic and UI layer and a database layer. When applications "used" DBF tables in one of two possible work-areas, the way applications accessed data was very tightly integrated into the FoxPro language. Nobody who was writing applications for PCs was thinking of the data layer as a separate "tier". But when you really think about it, "using a table" is not a vastly different concept from "connecting to a database server." Likewise, "setting a filter" is not all that different from "selecting a set of records." Developers simply fine-tuned these ideas, making their applications more scalable and more robust, but the core idea remained mostly unchanged.
Practically every Visual FoxPro application is deployed as an EXE that is the application. The EXE is an island of functionality and data management that runs locally on a client machine and connects to a database that either sits on the same machine, or somewhere on a server in a local area network. Many applications of the "COM and pre-COM era" share the same characteristics. With the invention of the Internet and mobile devices, application development and deployment much change.
Changing with the Times
Web applications represent the first step away from the more traditional application development model. People didn't want to have to install applications they used infrequently. For example, you don't want to install application on your local computer to become a customer of an eCommerce company such as Amazon.com. Instead, people simply want a light-weight tool like a Web browser. Visual FoxPro by itself does not play well in this scenario, but luckily, a number of third-party vendors rectified the situation. The companies behind the VFPConversion.com portal, including West Wind Technologies and EPS Software Corp, allowed Visual FoxPro developers to partake in the new world of "public applications." Admittedly, building HTML-based applications with Visual FoxPro isn't nearly as easy as building a traditional Windows applications, but tools such as Web Connection and the Voodoo Web Controls try hard to make it easier.
Another big change was that it no longer makes sense to build applications as "islands of functionality.". Instead, developers need to build applications as open systems. A typical Windows application may require accompanying Web interfaces. Applications may need to talk to each other, yet still provide traditional functionality and enforce traditional business rules. In other words, developers need to expose a system's functionality, not just allow database access. Businesses needed to build applications in a three-tiered fashion, with distinct layers for the database, application logic, and the UI.
Visual FoxPro and COM
Microsoft stepped up to the plate and added COM-support to Visual FoxPro, thereby enabling VFP to use the de-facto standard for application interoperability on the Windows platform. However, Visual FoxPro developers didn't adapt to using COM because developers weren't comfortable with the added complexity of the COM environment. In addition, an application built in Visual FoxPro that uses COM extensively imposes some productivity hurdles. For instance, Visual FoxPro doesn't have support for debugging a COM component.
In any case, Visual FoxPro can play in the HTML-world as well as the COM world. While a number of people embraced these technologies, many did not due to the involved complexities. As a custom software development company with a solid hold on both Visual FoxPro development and several other development technologies, EPS as a company sees many Visual FoxPro applications still in use today. Based on our experience, we find that the most common Visual FoxPro application in the wild today tend to be of the monolithic behemoth variety. These applications are very sophisticated, masterfully crafted, highly complex, but still based on principles popular in the days of FoxBase.
Inherent difficulties exist in trying to make Visual FoxPro function in a COM- and HTML-based world. If you do overcome that hurdle, you're left with an approach that has a set of problems on its own. COM is non-trivial in itself, but it becomes even more complex in large systems due to fundamental design problems such as sharing components across all applications running on a system. As you try to install and update users that will use your Visual FoxPro and COM solution, you'll encounter a phenomenon known as DLL hell. Also, COM does not work well in distributed systems. Microsoft tried to tackle this problem with DCOM, which allows you to instantiate objects that are installed on other machines in a network. This solution was cumbersome to use and very difficult to maintain. Worst of all, it did not work well in larger networks, particularly the Internet.
To make matters worse, application requirements continue to grow more and more complex. Having interconnected applications, potentially in distributed environments, raises application complexity exponentially. COM is simply not up to the task.
Enter the Smart Client Era
Of course, the changes didn't stop there. Beyond making applications more complex, users also expected applications to be more accessible and user friendly. "Why should my business application not be accessible when I'm away from the office," they wondered, "when the Internet connects everyone else?". Users want to run their business applications on their PDAs and cell phones (which is not possible with VFP since Microsoft has no intentions of porting Visual FoxPro to the Windows CE platform). Users want their applications to automatically integrate with other parts of their business.
Many developers tried to tackle some of these problems by providing HTML interfaces into many of their applications. In fact, there was a time when some people (including myself) argued that almost every application should be based on standard Internet technology to be more accessible. However, this approach is only the first step. You can use HTML for off-site use, but it is also less powerful and often performed painfully compared to Windows UIs, many users (generally the ones in the local office) frustrated. Also, HTML requires full connectivity to a Web service, which turns user's powerful computers into useless hardware while they are in airplanes or other places without connectivity.
So what does today's business application look like?
Not surprisingly, most modern business applications follow a common pattern: Data is stored in a secure and scalable database back end. Business logic is built in open technology that can then be accessed from practically any UI or other application. The business layer itself may be complex, but to the outside, a simple and clean "interface" is all that's needed. No fancy interconnectedness or complex webs of dependencies.
As mentioned before, the functionality provided by this system is exposed in a number of ways: Windows interfaces are used by the majority of users. This provides great power and performance. Of course, users expect these Windows applications to work across the Internet (such as from home) as well as in disconnected scenarios (such as in airplanes). Additionally, occasional users (for example, the boss who wants to run a report once a month), as well as public users (such as clients who intend to purchase a product), want to use an application without installing a client application. You can generally satisfy this need through an HTML-based interface, which provides either slightly different or limited access to the core application. You can also use this sort of interface for emergency situations where a user needs to get access to important data or functionality from an Internet café. It is difficult to make an HTML-based application as powerful and snappy as the Windows interface, but that's OK in these scenarios. And then, of course, you need to consider mobile devices. PocketPCs, cell phones, SmartPhones, and variations thereof. Surely, nobody wants to re-write a whole application just for small devices. Instead, you want to reuse all the business logic and exposed it through interfaces that your devices can access.
Another aspect that grows more and more important is the ability to expose applications to "non-human users." Some applications get used by other applications. The trade magazines call this Enterprise Integration or Business-to-Business. You shouldn't assume only large companies use these technologies. Even small applications can and should take advantage of integrating with services such as the Post Office or other public services. At the same time, these small applications should expose their functionality for tasks such as building different interfaces (perhaps for other platforms or devices) or to integrate with apps such as accounting packages. In many cases, locking functionality inside a single app is simply not a good business practice anymore and can cost your clients a lot of money.
Don't forget that your developers must make these applications secure and you must protect privacy interests. Applications also need to be of high quality, and flexible if the need for change arises. In these cases, applications need to be deployed quickly to potentially a very large number of users.
To make a long story short, here is a list of features a modern application requires:
Let's explore these requirements one by one.
Windows User Interfaces
HTML applications are great in many ways. Unfortunately, HTML does not measure up to the power of high-fidelity Windows UIs. This is true for environments such as Windows XP, and it is especially true for upcoming technologies such as the new Windows Longhorn (the codename for the next version of Windows) user experience.
Of course, Windows interfaces are only a small piece of the puzzle. Windows UI capabilities by themselves do not provide the power needed to build smart client application. The "return" of the Windows application is powered by technologies such as web services and disconnected data, which are discussed below.
How does Visual FoxPro measure up here? Well, you can use Visual FoxPro to build Windows interfaces. Visual FoxPro doesn't support many of the fancier modern techniques (even fundamental things such as tree controls), but you can add this functionality through ActiveX controls. This process works well, in general, although this approach loses many capabilities such as inheritance.
What's the outlook for the future? Uncertain at best. Few companies continue to develop ActiveX controls anymore, because "managed" languages (.NET languages) use a different standard. Visual FoxPro cannot use .NET controls, unfortunately, which slowly closes the door on the wealth of controls available. Also, future versions of Windows will use managed code to build interfaces. Microsoft will build the new Windows "Longhorn" UI layer on a standard currently known as XAML, which runs in the .NET environment. Microsoft has made no official announcements concerning Visual FoxPro and XAML support, but it would take a massive effort to convert VFP to support XAML. Posts by Microsoft officials in popular message boards confirm that Visual FoxPro 9.0 will remain in the COM world, and building applications that target Longhorn technologies (such as XAML, Avalon, WinFS,...) will require Visual Studio. At this point in time, I don't see a way to move a Visual FoxPro application to Longhorn in a way that would support Longhorn features.
HTML Interfaces
HTML has limitations, but it also has advantages. The fact that HTML will display in a browser rather than requiring a client application is significant. However, creating HTML interfaces - especially the more complex ones - is a tedious and labor intensive task. Of course, it is just as important to reuse code in HTML interfaces as it is Windows UIs. Unfortunately, HTML's string-based nature makes reusing code much harder. At EPS Software we have been trying to tackle this problem over time with tools such as Web Connection and the Voodoo Web Controls, which introduce an object-oriented aspect to web design. Similarly, ASP.NET is objectifying web development, which is a massive step forward.
How is VFP positioned here? Well, HTML is HTML no matter what tool you use to create it. ASP.NET is a much more powerful tool for creating HTML interfaces than Web Connection and Voodoo are. If you chose to use Visual FoxPro, tools like these (similar offerings are available from other companies) are your only option, but ASP.NET is starting to be an overwhelming competitor.
Where does this lead? ASP.NET will continue to push the envelope as the preeminent Web tool from Microsoft, both on the side of the technology (including available controls, objects, and fundamental plumbing), as well as in areas of developer productivity. Needless to say, Windows "Longhorn" will change the landscape drastically as XAML merges the advantages of Windows and HTML development, allowing users to effectively browse to XAML UIs as if they were Web pages.
Mobile Devices
Mobile devices such as PDAs and cell phones are important. Period. It often may not seem like that from a Visual FoxPro point of view, but this is clearly a case where the only tool Visual FoxPro developers have is a hammer, which makes screws look like a bad idea. In other words: If VFP could target mobile devices, programming for the millions of cell phones and PDAs out there might look like a great idea.
Imagine how large this market it. Millions of cell phones and PDAs are in use today. On top of that, there are other devices for specialty uses including barcode readers, in-vehicle computers, and more. This is a significant market with a lot of momentum that keeps growing in importance at a rapid pace.
Don't underestimate the power of these devices. It is true that they are not as powerful as full-featured PCs, but the range of applications that you can build for Pocket PCs is great. Anything from pocket Web sites (HTML apps for small devices) to smart client style applications (think Windows UI for small devices) are possible. Microsoft has even made versions of server style products such as Internet Information Server (IIS) and SQL Server available for Windows CE devices (such as Pocket PCs).
Microsoft has made Visual FoxPro's position in this market clear, both in the present and the future: Visual FoxPro can not target these devices. You must use either .NET or (depending on the device) a language such as C++ or Java. Clearly, .NET is the most productive choice.
Web Services and Message Based Programming
Web services are very important for modern applications. They allow applications to communicate across networks such as the Internet. If you are not familiar with Web services, you could imagine Web services as having the ability to instantiate an object across the Internet and call methods and access properties on it. For instance, an object that performs a certain business operation could be called through a Web service, which in essence replaces a conventional CreateObject() call. Or think of an object that provides data access. If that object can be called as a Web service, then automatically, the application using that service as part of its fundamental architecture can operate as a distributed application across the Internet.
Web services are pretty simple in nature. The standard for Web services is based on XML, which allows you to pass simple values or complex XML structures over the network. This means that you need to convert complex data, such as a database query result, to XML to transport it. This introduces some overhead unless the platform is fundamentally based on XML which is the case for .NET. Also, the objects that you use to build Web services need to be exposed in the Web service standard. Once again, this introduces some overhead both during development and during program execution.
Microsoft built .NET from the ground up with this sort of architecture in place. For instance, you could create an object that returns an ADO.NET DataSet (basically the equivalent of a Visual FoxPro cursor). Here's a simple conceptual example in VB .NET:
Public Class DataObject
Public Function GetData() As DataSet
' Create dataset here
Return DataSet
End Function
End Class
Now suppose that you want to access this as a local object. You could do this in the following fashion ("New" is the .NET equivalent of VFP's CreateObject()):
Dim dataObject As New DataObject
Dim cursor As DataSet
cursor = dataObject.GetData()
But what if you want to call the same object as a Web service across the Internet? I'll show you the code:
Dim dataObject As New DataObject
Dim cursor As DataSet
cursor = dataObject.GetData()
Can you spot the difference? That's right. There isn't one. .NET can automatically call the object as a Web service. Furthermore, .NET will deal with the details of transporting the DataSet across the Web service as well. For this reason, you can easily Internet-enable all .NET applications, even if they appear to be simple Windows applications. Note that this works in a lot of different environments. PocketPCs can call the same Web service as a Windows application or a Web application.
Also, you can use Web services as an interface layer on its own. Perhaps a partner organization, or a different application or division within the same company needs to access the same data. A Web service is a good solution for the problem.
Visual FoxPro can participate in Web service environments, although not as well. To create an object such as the DataObject above, you first create a Visual FoxPro object that can query data. You then have to convert the data to XML. You can use Visual FoxPro's XMLAdapter, but to help with this task, the developer still must do this task manually. Once you have that object, you can compile it as a COM object, which you can then import into ASP.NET. The developer can then create a manual .NET Web service wrapper that can expose the object.
On the client, Visual FoxPro developers can use the SOAP Toolkit to communicate with the Web service. in this case, your Visual FoxPro application calls the SOAP Toolkit as a COM object. This way, the Visual FoxPro developer can make a call to the GetData() method on the ASP.NET wrapped VFP COM object. This call returns the data as an XML string, which you can then convert to VFP usable data using the XMLAdapter object once again.
As you can imagine, most developers wouldn't want to do this as a fundamental architectural cornerstone of an application since you can easily do it in .NET. The process is simply too cumbersome. Creating and maintaining this sort of architecture in Visual FoxPro becomes too cumbersome due to the complexity and due to the lack of fundamental features such as debugging abilities. And then, of course, this solution has a performance problem due to the overhead.
Realistically speaking, Visual FoxPro can create Web services for the purpose of enterprise integration. VFP can also consume Web services that someone else may have made available. However, no one recommends embracing Web services with Visual FoxPro as an architectural cornerstone of an application.
As you know, true distributed applications generally need additional features to provide a satisfying user experience. For example, calling a Web service over a dialup connection may be rather slow. The application will still work, but it isn't really professional to make an application appear hung for a few minutes while it waits for data to download? Most likely, you want more polish in your solution. .NET provides this sort of polish through techniques such as multi-threading, which Visual FoxPro does not support.
Disconnected Data
Pre-.NET data access is generally done in a connected fashion. In SQL Server scenarios for instance, your application opens a connection to SQL Server, it modifies data, your application writes data back to the server, and then your application closes the connection. In Visual FoxPro your application opens a table, users modify data, and finally, the table gets closed. You can use this approach in monolithic or client/server applications, but it does not work in truly distributed applications. For instance, the data that returns from a Web service must be able to exist entirely separate from the back end server it came from.
Creating disconnected data in environments such as Visual FoxPro is technically possible. You can move single records into memory using scatter/gather commands. Alternatively, you can use cursors to copy to temp files, or you can turn them into XML. However, you must do a significant amount of extra work to create this functionality. In .NET, on the other hand, you must automatically handle all data in a disconnected fashion, effectively eliminating this issue altogether.
From an architectural point of view, "disconnected data" also includes a number of other issues, most importantly the ability to store data for offline use. This is similar to Outlook's offline email store that Outlook uses when a connection to Exchange is not available. Implementing this functionality requires the ability to tie into the data flow in a way that easily allows caching data offline. Secondly, your application needs the ability to store that data in some temporary data store.
In terms of storing data, Visual FoxPro features one of the greatest file-based data engines ever built. In .NET your solution either needs to store XML or use a non-native database technology such as MSDE. This generally is not a big problem, but clearly, Visual FoxPro is better at this task. You can't easily use Visual FoxPro to solve the problem of intercepting all data for caching purposes though. .NET, on the other hand, makes it relatively straightforward to look at all data returned from data access objects and store it away for later usage. This feature is not natively built into .NET, but its architecture makes it easy to implement such a feature.
Security
Security is a significant issue. As developers, you have the fundamental responsibility toward the user of respecting their privacy (including the data they entrust to you) and you must make sure that you treat their security seriously. Anything else is simply not acceptable.
Security is every developer's responsibility. Many people think that security is a problem that companies like Microsoft need to solve, but this is not the case. Just like every line of code written has the possibility of introducing a bug, every line of code written has the possibility of introducing a security vulnerability. You don't add security as a feature to an application. Instead, every single line of code, and every single design element of an application is either secure, or it is not. (Or more accurately, every line has a certain degree of security.)
Often, people ask me why they should worry about security. After all, who would target them individually? What do they have that an attacker might want? My reply is: Why would an attacker exclude a single person? In a day and age where teenagers can download simple scripts that probe every single computer attached to the Internet in a matter of days, why would they specifically exclude you? Even if they wanted to, they probably couldn't do so because such code is so automated! The fact of the matter is that every single computer that's online gets probed for security holes several times a day. (For more information on this subject, I recommend reading "Writing Secure Code, ISBN 0-7356-1722-8").
You can increase an application's security in a number of ways. You can use a secure database such as SQL Sever. Furthermore, you should run your applications on a secure platform such as the .NET Runtime (CLR), which has code security mechanisms built in at the lowest levels. All code that .NET executes is governed by "evidence" and applied policies. For instance, code loaded from a Web server automatically has a smaller set of rights than code loaded from the local machine. Furthermore, developers can easily make sure code only has the right to execute limited kinds of operations, limiting the ability of a hacker's code to change that operation. Additionally, the CLR protects against many common security problems such as buffer overruns.
In contrast, Visual FoxPro has no security features. You can choose to use some limited options to restrict access to the database, but those are along the lines of business logic. No mechanisms exist to discourage attackers. In fact, you can use many DBF tools to open a FoxPro database. Also, no code-level security mechanisms exist in Visual FoxPro. Additionally, consider data safety. Visual FoxPro data is prone to corruption, an issue that is practically unknown with SQL Server.
Application Deployment
Deployment is a very important issue for modern applications. Rolling out frequent updates to even as few as 10 people can be a major problem and a serious expense.
.NET has a number of great solutions to the deployment dilemma. .NET lets you create deployment mechanisms that do not have to be installed in the same fashion that pre-.NET apps have to be installed. For some .NET applications, you can simply cop a file to a location and that's all of the required deployment. You don't need to register DLLs. Using .NET you have fewer compatibility issues since each application is independent from other parts of the system. In fact, .NET can maintain multiple versions of the same component if needed. Using .NET you can also create self-updating applications because Microsoft provides code to perform those task.
.NET supports a number of alternate options that allow the developer to deploy only to selected servers. Updating single servers is a lot more manageable than updating a potentially large number of clients.
Visual Studio 2005 will feature a number of improvements for deployment such as completely automated "ClickOnce" deployment of Windows applications, allowing users to browse to a Windows application with a Web browser, which simply installs, launches, and later updates the application completely transparently and safely.
Visual FoxPro provides no deployment features other than creating Windows setup programs that individual users can execute.
Another important aspect of the deployment puzzle is version control. In the COM-era, many applications broke when other (seemingly unrelated) components were installed. .NET fixes this problem through integrated version control. VFP also has no concept of version control.
Conclusion
Visual FoxPro is an excellent tool that I have used for close to 15 years, and I will continue to use it for a while longer. I think Visual FoxPro is a great tool for building monolithic database applications. If your business needs are such that you can continue to build such applications, then stick to VFP! I encourage you to embrace new technologies such as SQL Server so that your data is more secure, easier to backup, and less likely to corrupt, but do not abandon VFP as a platform.
If, however, you want to build distributed enterprise applications I describe in this paper, and if you want to build applications that you can upgrade to environments such as Windows "Longhorn", then .NET is a better choice! If you are investing a significant amount of time, effort, or money into a project, you should consider how you'll deal with users and servers that will use upcoming versions of Windows. If you sell off-the-shelf applications, your customers rely heavily on perceived or real longevity. You should always assume that custom applications that you deploy within a single organization generally might need to be maintained for several years. Both scenarios are likely to benefit greatly from the .NET environment.
Visual FoxPro does not fair well in many of the demands put forward by modern business applications (as outlined in this paper).Whether you want to stick with VFP, add .NET to your toolbox, or completely move to .NET, every developer must make this decision based on the scenario he or she is facing. In the past, EPS Software Corporation has provided great information about Visual FoxPro and we will continue to do so. However, if you are interested in moving to .NET, then VFPConversion.com will be available to you as a free source of information and guidance.
Markus Egger
President and Chief Software Architect
EPS Software Corp.
