On the F# team at Microsoft, we're constantly improving the F# language to empower developers to do functional programming on .NET. Over the previous four releases, from 2017 until now, we've been on a long journey to make F# awesome on .NET Core. We've revamped the F# compiler and core library to run cross-platform, added support for Span<T> and low-level, cross-platform programming, and added the ability to preview language features that can ship with .NET preview releases.

With the .NET 5 release, we're releasing F# 5, the next major version of the F# language. But F# 5 isn't just a bundle of features that comes along for the ride with .NET 5. F# 5 marks the end of the current era ? bringing up support for .NET Core ? and the beginning of a new one. With F# 5, we're considering our journey to bring F# to .NET Core mostly complete. With F# 5, our focus shifts from .NET Core to three major areas:

• Interactive programming
• Making analytical-oriented programming convenient and fun
• Great fundamentals and performance for functional programming on .NET

In this article, I'll go through the F# language and tooling features we've implemented for F# 5 and explain how they align with our goals.

F# 5 Makes Interactive Programming a Joy

F# has a long history of being interactive. In fact, when F# 1.0 was developed, a tool called F# Interactive (FSI) was developed for the eventual release of F# 1.0 in 2006. This coincided with the first tooling integration into Visual Studio. FSI was used quite heavily in the initial marketing of F# (as shown in Figure 1) to demonstrate iterative and interactive development of Windows Forms applications, graphics scenes, and games on Windows.

Figure 1: Initial prototype of F# Interactive in Visual Studio 2005

The core experiences of FSI have largely remained the same in F# 5. These include:

• The ability to reference and call into assemblies on your computer
• The ability to load other F# scripts to execute as a collection of scripts
• Integration with Visual Studio
• The ability to customize output

However, as F# and the .NET ecosystem moved from assemblies on a computer to packages installed via a package manager, many F# developers using FSI for various tasks found themselves annoyed by having to manually download a package and reference its assemblies manually. Additionally, as .NET's reach extended beyond Windows, developers on macOS or Linux found themselves missing features and relying on a Mono installation to use FSI in their environments.

Introducing Package Management Support in FSI

Using a package in an F# script has long been a source of frustration for F# programmers. They typically downloaded packages themselves and referenced assemblies in the path to the package manually. A smaller set of F# programmers used the Paket package manager and generated a “load script” ? a feature in Paket that generates an F# script file with references to all the assemblies in the packages you want to reference ? and loads this script into their working F# scripts. However, because Paket is an alternative to NuGet instead of a default tool, most F# programmers don't use it.

Now with F# 5, you can simply reference any NuGet package in an F# script. FSI restores this package with NuGet and automatically references all assemblies in the package. Here's an example:

#r "nuget: Newtonsoft.Json"

open Newtonsoft.Json

let o = {| X = 2; Y = "Hello" |}
printfn "%s" (JsonConvert.SerializeObject o)

When you execute the code in that snippet, you'll see the following output:

{"X":2,"Y":"Hello"}
val o : {| X: int; Y: string |} = { X = 2 Y = "Hello" }
val it : unit = ()

The package management feature can handle just about anything you want to throw at it. It supports packages with native dependencies like ML.NET or Flips. It also supports packages like FParsec, which previously required that each assembly in the package is referenced in a specific order in FSI.

Introducing dotnet FSI

The second major frustration for F# programmers using FSI is that it was missing in .NET Core for a long time. Microsoft released an initial version of FSI for .NET Core with .NET Core 3.0, but it was only useful for F# scripts that didn't incorporate any dependencies. Now, in conjunction with package management, you can use FSI for all the same tasks on macOS or Linux as you would on Windows (except for launching WinForms and WPF apps, for obvious reasons). This is done with a single command: dotnet fsi.

Introducing F# Support in Jupyter Notebooks

There's no question that package management and making FSI available everywhere makes F# better for interactive programming. But Microsoft felt that we could do more than just that. Interactive programming has exploded in recent years in the Python community, thanks in large part to Jupyter Notebooks. The F# community had built initial support for F# in Jupyter many years ago, so we worked with its current maintainer to learn about what a good experience for Jupyter meant and built it.

Now, with F# 5, you can pull in packages, inspect data, and chart the results of your experimentation in a sharable, cross-platform notebook that anyone can read and adjust, as shown in Figure 2.

Figure 2: Charting in Jupyter notebooks with F#

Another reason why we're very excited about F# support in Jupyter Notebooks is that the notebooks are easy to share with other people. Juputer Notebooks render as markdown documents in GitHub and other environments. Not only are they a programming tool, but they produce a document that can be used to instruct others how to perform tasks, share findings, learn a library, or even learn F# itself!

Introducing F# Support in Visual Studio Code Notebooks

F# support in Jupyter Notebooks brings interactivity to a whole new level. But Jupyter isn't the only way to program for a notebook. Visual Studio Code is also bringing notebook programming into the fold, with all the power of a language service that you would expect to find when editing code in a normal file. With F# support in Visual Studio Code Notebooks, you can enjoy language service integration when building a notebook, as shown in Figure 3.

Figure 3: Installing a package and using it in Visual Studio Code Notebooks

Another benefit of Visual Studio Code notebooks is its file format, which is designed to be human-readable and easy to diff in source control. It supports importing Jupyter Notebooks and exporting Visual Studio Code notebooks as Jupyter Notebooks, as you can see in Figure 4.

Figure 4: Converting a Jupyter Notebook in Visual Studio Code

You can do many things with F# in Visual Studio Code and Jupyter Notebooks, and we're looking to expand the capabilities beyond what's been described so far. Our roadmap includes integration with various other tools, more cohesive data visualization, and data interop with Python.

F# 5 Lays More Foundations for Analytical Programming

A paradigm of growing importance in the age of ubiquitous machine learning and data science is what I like to call “analytical programming.” This paradigm isn't exactly new, although there are new techniques, libraries, and frameworks coming out every day to further advance the space. Analytical programming is all about analyzing and manipulating data, usually applying numerical techniques to deliver insights. This ranges from importing a CSV and computing a linear regression on the data to the most advanced and compute-intensive neural networks coming out of AI research institutions.

F# 5 represents the beginning of our foray into this space. The team at Microsoft thinks that F# is already great for manipulating data, as countless F# users have demonstrated by using F# for exactly that purpose. F# also has great support for numeric programming with some built-in types and functions and a syntax that's approachable and succinct. So we kept that in mind and identified some more areas to improve.

Consistent Behavior with Slices

A very common operation performed in analytical programming is taking a slice of a data structure, particularly arrays. F# slices used to behave inconsistently, with some out-of-bounds behavior resulting in a runtime exception and others resulting in an empty slice. We've changed all slices for F# intrinsic types - arrays, lists, strings, 3D arrays, and 4D arrays - to return an empty slice for any slice you might specify that couldn't possibly exist:

let l = [ 1..10 ]
let a = [| 1..10 |]
let s = "hello!"

// Before: empty list
// F# 5: same
let emptyList = l.[-2..(-1)]

// Before: would throw exception
// F# 5: empty array
let emptyArray = a.[-2..(-1)]

// Before: would throw exception
// F# 5: empty string
let emptyString = s.[-2..(-1)]

The reasoning for this is largely because in F#, empty slices compose with nonempty slices. An empty string can be added to a nonempty string, empty arrays can be appended to nonempty arrays, etc. This change is non-breaking and allows for predictability in behavior.

Fixed Index Slicing for 3D and 4D Arrays

F# has built-in support for 3D and 4D arrays. These array types have always supported slicing and indexing, but never slicing based on a fixed index. With F# 5, this is now possible:

// First, create a 3D array with values from 0 to 7
let dim = 2
let m = Array3D.zeroCreate<int> dim dim dim
let mutable cnt = 0
for z in 0..dim-1 do
    for y in 0..dim-1 do
        for x in 0..dim-1 do
            m.[x,y,z] <- cnt
            cnt <- cnt + 1

// Now let's get the [4;5] slice!
m.[*, 0, 1]

This helps complete the picture for slicing scenarios with 3D and 4D arrays.

Preview: Reverse Indexes

Microsoft is also introducing the ability to use reverse indexes, which can be used with slices, as a preview in F# 5. To use it, simply place <LangVersion>preview</LangVersion> in your project file.

let xs = [1..10]

// Get element 1 from the end:
xs.[^1]

// Old way to get the last two elements
let lastTwoOldStyle = xs.[(xs.Length-2)..]

// New way to get the last two elements
let lastTwoNewStyle = xs.[^1..]

lastTwoOldStyle = lastTwoNewStyle // true

You can also define your own members via an F# type extension to augment these types to support F# slicing and reverse indexes. The following example does so with the Span<'T> type:

open System
type Span<'T> with
    member sp.GetSlice(startIdx, endIdx) =
        let s = defaultArg startIdx 0
        let e = defaultArg endIdx sp.Length
        sp.Slice(s, e - s)

    member sp.GetReverseIndex(_, offset: int) =
        sp.Length - offset
        
let sp = [| 1; 2; 3; 4; 5 |].AsSpan()
sp.[..^2] // [|1; 2; 3|]

F# intrinsic types have reverse indexes built in. In a future release of F#, we'll also support full interop with System.Index and System.Range, at which point, the feature will no longer be in preview.

Enhanced Code Quotations

F# Code Quotations are a metaprogramming feature that allows you to manipulate the structure of F# code and evaluate it in an environment of your choosing. This capability is essential for using F# as a model construction language for machine learning tasks, where the AI model may run on different hardware, such as a GPU. A critical piece missing in this puzzle has been the ability to faithfully represent F# type constraint information, such as those used in generic arithmetic, in the F# quotation so that an evaluator can know to apply those constraints in the environment it's evaluating in.

Starting with F# 5, constraints are now retained in code quotations, unlocking the ability for certain libraries such as DiffSharp to use this part of the F# type system to its advantage. A simple way to demonstrate this is the following code:

open FSharp.Linq.RuntimeHelpers

let eval q =
    LeafExpressionConverter
        .EvaluateQuotation q

let inline negate x = -x

// Crucially, 'negate' has
//       the following signature:
//
// val inline negate:
//         x: ^a -> ^a
//                    when  ^a:
//                      (static member ( ~- ):  ^a ->  ^a)
//
// This constraint is critical to F# type safety
//       and is now retained in quotations      .
<@ negate 1.0 @>  |> eval

The use of an arithmetic operator implies a type constraint such that all types passed to negate must support the ? operator. This code fails at runtime because the code quotation doesn't retain this constraint information, so evaluating it throws an exception.

Code quotations are the foundation for some more R&D-heavy work being done to use F# as a language for creating AI models, and so the ability to retain type constraint information in them helps make F# a compelling language for programmers in this space who seek a little more type safety in their lives.

F# 5 Has Great Fundamentals

F# 5 may be about making interactivity and analytical programming better, but at its core, F# 5 is still about making everyday coding in F# a joy. F# 5 includes several new features that both app developers and library authors can enjoy.

Support for nameof

First up is a feature that C# developers have come to love: nameof. The nameof operator takes an F# symbol as input and produces a string at compile-time that represents that symbol. It supports just about all F# constructs. The nameof operator is often used for logging diagnostics in a running application.

#r "nuget: FSharp.SystemTextJson"

open System.Text.Json
open System.Text.Json.Serialization
open System.Runtime.CompilerServices

module M =
    let f x = nameof x
    
printfn "%s" (M.f 12)
printfn "%s" (nameof M)
printfn "%s" (nameof M.f)

/// Simplified version of EventStore's API
type RecordedEvent =
    { EventType: string 
      Data: byte[] }
      
/// My concrete type:
type MyEvent =
    | AData of int
    | BData of string
    
// use 'nameof' instead of the string literal in the match expression
let deserialize (e: RecordedEvent) : MyEvent =
    match e.EventType with
    | nameof AData ->
        JsonSerializer.Deserialize<AData> e.Data
        |> AData
    | nameof BData ->
        JsonSerializer.Deserialize<BData> e.Data
        |> BData
    | t -> failwithf "Invalid EventType: %s" t

Interpolated Strings

Next is a feature seen in languages such as C# and JavaScript: Interpolated Strings. Interpolated strings allow you to create interpolations or holes in a string that you can fill in with any F# expression. F# interpolated strings support typed interpolations synonymous with the same format specifies in sprintf and printf strings formats. F# interpolated strings also support triple-quotes strings. Just like in C#, all symbols in an F# interpolation are navigable, able to be renamed, and so on.

// Basic interpolated string
let name = "Phillip"
let age = 29
let message = $"Name: {name}, Age: {age}"

// Typed interpolation
// '%s' requires the interpolation to be a string
// '%d' requires the interpolation to be an int
let message2 = $"Name: %s{name}, Age: %d{age}"

// Verbatim interpolated strings
// Note the string quotes allowed inside the
// interpolated string
let messageJson = $"""
"Name": "{name}",
"Age": {age}"""

Additionally, you can write multiple expressions inside interpolated strings, producing a different value for the interpolated expression based on an input to the function. This is a more of a niche use of the feature, but because any interpolation can be a valid F# expression, it allows for a great deal of flexibility.

Open Type Declarations

F# has always allowed you to open a namespace or a module to expose its public constructs. Now, with F# 5, you can open any type to expose static constructs like static methods, static fields, static properties, and so on. F# union and records can also be opened. You can also open a generic type at a specific type instantiation.

open type System.Math

let x = Min(1.0, 2.0)

module M =
    type DU = A | B | C
    
    let someOtherFunction x = x + 1
    
// Open only the type inside the module
open type M.DU

printfn "%A" A

Enhanced Computation Expressions

Computation expressions are a well-loved set of features that allow library authors to write expressive code. For those versed in category theory, they are also the formal way to write Monadic and Monoidal computations. F# 5 extends computation expressions with two new features:

• Applicative forms for computation expressions via let!..and! keywords
• Proper support for overloading Custom Operations

“Applicative forms for computation expressions” is a bit of a mouthful. I'll avoid diving into category theory and instead work through an example:

// First, define a 'zip' function
module Result =
    let zip x1 x2 =
        match x1,x2 with
        | Ok x1res, Ok x2res ->
            Ok (x1res, x2res)
        | Error e, _ -> Error e
        | _, Error e -> Error e
        
// Next, define a builder with 'MergeSources' and 'BindReturn'
type ResultBuilder() =
    member _.MergeSources(t1: Result<'T,'U>,
                          t2: Result<'T1,'U>) =
        Result.zip t1 t2
    member _.BindReturn(x: Result<'T,'U>, f) =
        Result.map f x
        
let result = ResultBuilder()

let run r1 r2 r3 =
    // And here is our applicative!
    let res1: Result<int, string> =
        result {
            let! a = r1
            and! b = r2
            and! c = r3
            return a + b - c
        }

    match res1 with
    | Ok x ->
        printfn "%s is: %d" (nameof res1) x
    | Error e ->
        printfn "%s is: %s" (nameof res1) e

Prior to F# 5, each of these and! keywords would have been let! keywords. The and! keyword differs in that the expression that follows it must be 100% independent. It cannot depend on the result of a previous let!-bound value. That means code like the following fails to compile:

let res1: Result<int, string> =
    result {
        let! a = r1
        and! b = r2 a // try to pass 'a'
        and! c = r3 b // try to pass 'b'
        return a + b - c
    }

So, why would we make that code fail to compile? A few reasons. First, it enforces computational independence at compile-time. Second, it does buy a little performance at runtime because it allows the compiler to build out the call graph statically. Third, because each computation is independent, they can be executed in parallel by whatever environment they're running in. Lastly, if a computation fails, such as in the previous example where one may return an Error value instead of an Ok value, the whole thing doesn't short-circuit on that failure. Applicative forms “gather” all resulting values and allow each computation to run before finishing. If you were to replace each and! with a let!, any that returned an Error short-circuits out of the function. This differing behavior allows library authors and users to choose the right behavior based on their scenario.

If this sounds like it's a little concept-heavy, that's fine! Applicative computations are a bit of an advanced concept from a library author's point of view, but they're a powerful tool for abstraction. As a user of them, you don't need to know all the ins and outs of how they work; you can simply know that each computation in a computation expression is guaranteed to be run independently of the others.

Another enhancement to computation expressions is the ability to properly support overloading for custom operations with the same keyword name, support for optional arguments, and support for System.ParamArray arguments. A custom operation is a way for a library author to specify a special keyword that represents their own kind of operation that can happen in a computation expression. This feature is used a lot in frameworks like Saturn to define an expressive DSL for building Web apps. Starting with F# 5, authors of components like Saturn can overload their custom operations without any caveats, as shown in Listing 1.

Listing 1: Computation Expressions can overload custom operations

type InputKind =
    | Text of placeholder:string option
    | Password of placeholder: string option

type InputOptions =
    { Label: string option
      Kind: InputKind
      Validators: (string -> bool) array }

type InputBuilder() =
    member t.Yield(_) =
        { Label = None
          Kind = Text None
          Validators = [||] }

[<CustomOperation("text")>]
member this.Text(io,?placeholder) =
    { io with Kind = Text placeholder }

[<CustomOperation("password")>]
member this.Password(io,?placeholder) =
    { io with Kind = Password placeholder }

[<CustomOperation("label")>]
member this.Label(io,label) =
    { io with Label = Some label }

[<CustomOperation("with_validators")>]
member this.Validators(io, [<System.ParamArray>] validators) =
    { io with Validators = validators }

let input = InputBuilder()

let name =
    input {
    label "Name"
    text
    with_validators
        (String.IsNullOrWhiteSpace >> not)
    }

let email =
    input {
    label "Email"
    text "Your email"
    with_validators
        (String.IsNullOrWhiteSpace >> not)
        (fun s -> s.Contains "@")
    }

let password =
    input {
    label "Password"
    password "Must contains at least 6 characters, one number and one uppercase"
    with_validators
        (String.exists Char.IsUpper)
        (String.exists Char.IsDigit)
        (fun s -> s.Length >= 6)
    }

Proper support for overloads in Custom operations are developed entirely by two F# open source contributors Diego Esmerio and Ryan Riley.

With applicative forms for computation expressions and the ability to overload custom operations, we're excited to see what F# library authors can do next.

Interface Implementations at Different Generic Instantiations

Starting with F# 5, you can now implement the same interface at different generic instantiations. This feature was developed in partnership with Lukas Rieger, an F# open source contributor.

type IA<'T> =
    abstract member Get : unit -> ?T

type MyClass() =
    interface IA<int> with
        member x.Get() = 1
    interface IA<string> with
        member x.Get() = "hello"

let mc = MyClass()
let asInt = mc :> IA<int>
let asString = mc :> IA<string>

asInt.Get() // 1
asString.Get() // "hello"

More .NET Interop Improvements

.NET is an evolving platform, with new concepts introduced every release and thus, more opportunities to interoperate. Interfaces in .NET can now specify default implementations for methods and properties. F# 5 lets you consume these interfaces directly. Consider the following C# code:

using System;

namespace CSharpLibrary
{
    public interface MyDim
    {
        public int Z => 0;
    }
}

This interface can be consumed directly in F#:

open CSharp

// Create an object expression to implement the interface
let md = { new MyDim }
printfn $"DIM from C#: {md.Z}"

Another concept in .NET that's getting some more attention is nullable value types (formerly called Nullable Types). Initially created to better represent SQL data types, they are also foundational for core data manipulation libraries like the Data Frame abstraction in Microsoft.Data.Analysis. To make it a little easier to interop with these libraries, you apply a new type-directed rule for calling methods and assigning values to properties that are a nullable value type. Consider the following sample using this package with a package reference directive:

#r "nuget: Microsoft.Data.Analysis"

open System
open Microsoft.Data.Analysis

let dateTimes =
    "Datetimes"
    |> PrimitiveDataFrameColumn<DateTime>

// The following used to fail to compile
Let date = DateTime.Parse("2019/01/01")
dateTimes.Append(date)

// The previous is now equivalent to      :
Let date = DateTime.Parse("2019/01/01")
Let data = Nullable<DateTime>(date)
dateTimes.Append(data)

These examples used to require that you explicitly construct a nullable value type with the Nullable type constructor as the example shows.

Better Performance

The Microsoft team has spent the past year improving F# compiler performance both in terms of throughput and tooling performance in IDEs like Visual Studio. These performance improvements have rolled out gradually rather than as part of one big release. The sum of this work that culminates in F# 5 can make a difference for everyday F# programming. As an example, I've compiled the same codebase ? the core project in FSharpLus, a project that notoriously stresses the F# compiler ? three times. Once for F# 5, once for the latest F# 4.7 with .NET Core, and once for the latest F# 4.5 in .NET Core, as shown in Table 1.

The results in Table 1 come from running dotnet build /clp:PerformanceSunnary from the command-line and looking at the total time spent in the Fsc task, which is the F# compiler. Results might vary on your computer depending on things like process priority or background work, but you should see roughly the same decreases in compile times.

IDE performance is typically influenced by memory usage because IDEs, like Visual Studio, host a compiler within a language service as a long-lived process. As with other server processes, the less memory you use up, the less GC time is spent cleaning up old memory and the more time can be spent processing useful information. We focused on two major areas:

• Making use of memory-mapped files to back metadata read from the compiler
• Re-architecting operations that find symbols across a solution, like Find All References and Rename

The result is significantly less memory usage for larger solutions when using IDE features. Figure 5 shows an example of memory usage when running Find References on the string type in FAKE, a very large open source codebase, prior to the changes we made.

Figure 5: Peak memory usage running Find References on string in FAKE.sln in VS 16.5

This operation also takes one minute and 11 seconds to complete when run for the first time.

With F# 5 and the updated F# tools for Visual Studio, the same operation takes 43 seconds to complete and uses over 500MB less memory, as shown in Figure 6.

Figure 6: Peak memory usage running Find References on string in FAKE.sln in VS 16.6 and higher

The example with results shown in Figure 5 and Figure 6 is extreme, since most developers aren't looking for usages of a base type like string in a very large codebase, but it goes to show how much better performance is when you're using F# 5 and the latest tooling for F# compared to just a year ago.

Performance is something that is constantly worked on, and improvements often come from our open source contributors. Some of them include Steffen Forkmann, Eugene Auduchinok, Chet Hust, Saul Rennison, Abel Braaksma, Isaac Abraham, and more. Every release features amazing work by open source contributors; we're eternally grateful for their work.

The Continuing F# Journey and How to Get Involved

The Microsoft F# team is very excited to release F# 5 this year and we hope you'll love it as much as we do. F# 5 represents the start of a new journey for us. Looking forward, we're going to continually improve interactive experiences to make F# the best choice for notebooks and other interactive tooling. We're going to go deeper in language design and continue to support libraries like DiffSharp to make F# a compelling choice for machine learning. And as always, we're going to improve on F# compiler and tooling fundamentals and incorporate language features that everyone can enjoy.

We'd love to see you come along for the ride, too. F# is entirely open source, with language suggestions, language design, and core development all happening on GitHub. There are some excellent contributors today and we're seeking out more contributors who want to have a stake in how the F# language and tools evolve moving forward.

To get involved on a technical level, check out the following links:

• F# language suggestions: https://github.com/fsharp/fslang-suggestions
• F# language design: https://github.com/fsharp/fslang-design
• F# development: https://github.com/dotnet/fsharp
• F# running on JavaScript: https://fable.io/
• F# tooling for Visual Studio Code: http://ionide.io/
• F# running on Web Assembly: https://fsbolero.io/

The F# Software Foundation also hosts a large Slack community, in addition to being a central point for various sub-communities to share information with one another. It's free to join, so head over to the website here to learn more: http://foundation.fsharp.org/join

Want to have a say in where F# goes next and how it does it? Come join us. We'd love to work together.