I am joining Cloud AI team to work on Azure Search

Azure Search

It has been over 3 years since I joined the Azure Portal team. During that time I learned a lot about every aspect of web and mobile development. I delivered over 20 technical talks at different conferences around the World and local meetups. It was amazing to take the new Portal from preview to v1. In the meantime, during the //oneweek hackathon, together with a few other folks, we built a prototype of the Azure Mobile App. After getting feedback from Scott Guthrie who said that “it would be super useful” I started working on the app overnight.

I didn’t know much about mobile development at the time, but I wanted to learn. I didn’t know much about complexities of Active Directory authentication and Azure Resource Manager APIs. I just knew that it would be super cool to have an app that would allow me to check the status of my Azure resources while waiting for my lunch. Receiving a push notification, and being able to scale VM from my phone would be also tremendously valuable.

When I started working on the app full time, my dream came true. I could truly connect my passion with work. I enjoyed the long hours, and late nights we all put to make it happen. The day when Scott Hanselman presented the Azure App at the //build conference was on of the best days of my life.

Now, when the Azure App is released, and backed by great team, I can move to the next challenge.

Machine learning is becoming part of every aspect of our lives. Over last few years, ML crossed a threshold necessary to be extremely useful. I always wanted to be part of it. I took a great Coursera class by Andrew Ng, I started overnight project StockEstimator and I got involved in SeeingAI to learn how Real-World Machine Learning looks like.

Now, I’m taking it to the next level. I am joining Azure Search Team to lead their User Experience. I will be responsible for bringing the product to customers. While using my existing web development knowledge, I will have an amazing opportunity to learn more about Big Data, AI and ML.

Azure Search is managed cloud search service that offers scalable full-text search over multiple languages, geo-spatial search, filtering and faceted navigation, type-ahead queries, hit highlighting, and custom analyzers. You can find more details in this talk by Pablo Castro (Azure Search manager and creator of Open Data Protocol).

The cool thing about working for Microsoft is that you may end up working with person who created HTTP protocol. Henrik Frystyk Nielsen, former Tim Berners-Lee’s student, who shared office with Håkon Wium Lie (creator of CSS), joined my new team this month. What’s even cooler, he is sitting next to me 🙂

In my new office with Henrik:

Henrik Frystyk Nielsen and Jacob Jedryszek

If you want to learn more about all the cool stuff we are doing at Cloud AI group there is an awesome .NET Rocks Podcast with Joseph Sirosh. Check it out!

There is also awesome talk by Joseph from the last Connect(); conference, which includes JFK files demo presented by Corom Thompson from my team (creator of How-Old.NET). In that demo Corom showcases how you can use Azure Search and Cognitive Services to explore JFK files. Super cool! You can see demo in below video, and code on github.

It has never been a better time to work on the intersection of Cloud and Artificial Intelligence!


Adding biometrics authentication to Xamarin.iOS (Touch ID / Face ID) and Xamarin.Android (Fingerprint)

One of the top Azure App users requests was to add Touch ID support for additional security. In this post I will share the details of implementing biometrics authentication for iOS and Android with Xamarin.

There are three aspects of biometrics auth:
1. Enable user to turn biometrics authentication on and off. Users shouldn’t be forced to use this additional security feature.
2. Detecting when user should be asked for biometrics authentication, e.g., when app is coming from background, and when app is starting.
3. Authentication process. Includes detecting hardware capabilities (is touch or face id available?), and local setup (does user configured local authentication in system settings).

Enabling biometrics authentication usually can be controlled in settings (like in Outlook or OneDrive). We did the same in Azure App:

Require Touch ID Settings

iOS

Detecting when user is switching back to our app in iOS is pretty simple. Every time when user switch from background, method WillEnterForeground in AppDelegate is being called. We just need to override it with our custom implementation:

public override void WillEnterForeground(UIApplication application)
{
    // biometrics authentication logic here
}

You should also authenticate user when app is being launched. In that case authentication should be performed in your initial view controller.

In iOS we have 2 kinds of biometrics authentication:
1. Touch ID
2. Face ID (available from iPhoneX)

We can also fallback to passcode if touch/face ID is not configured, or user’s device does not support it.

The iOS Local Auth API is pretty straightforward, and well documented. I created simple helper to handle feature detection and authentication:

public static class LocalAuthHelper
{
    private enum LocalAuthType
    {
        None,
        Passcode,
        TouchId,
        FaceId
    }

    public static string GetLocalAuthLabelText()
    {
        var localAuthType = GetLocalAuthType();

        switch (localAuthType)
        {
            case LocalAuthType.Passcode:
                return Strings.RequirePasscode;
            case LocalAuthType.TouchId:
                return Strings.RequireTouchID;
            case LocalAuthType.FaceId:
                return Strings.RequireFaceID;
            default:
                return string.Empty;
        }
    }

    public static string GetLocalAuthIcon()
    {
        var localAuthType = GetLocalAuthType();

        switch (localAuthType)
        {
            case LocalAuthType.Passcode:
                return SvgLibrary.LockIcon;
            case LocalAuthType.TouchId:
                return SvgLibrary.TouchIdIcon;
            case LocalAuthType.FaceId:
                return SvgLibrary.FaceIdIcon;
            default:
                return string.Empty;
        }
    }

    public static string GetLocalAuthUnlockText()
    {
        var localAuthType = GetLocalAuthType();

        switch (localAuthType)
        {
            case LocalAuthType.Passcode:
                return Strings.UnlockWithPasscode;
            case LocalAuthType.TouchId:
                return Strings.UnlockWithTouchID;
            case LocalAuthType.FaceId:
                return Strings.UnlockWithFaceID;
            default:
                return string.Empty;
        }
    }

    public static bool IsLocalAuthAvailable => GetLocalAuthType() != LocalAuthType.None;

    public static void Authenticate(Action onSuccess, Action onFailure)
    {
        var context = new LAContext();
        NSError AuthError;

        if (context.CanEvaluatePolicy(LAPolicy.DeviceOwnerAuthenticationWithBiometrics, out AuthError)
            || context.CanEvaluatePolicy(LAPolicy.DeviceOwnerAuthentication, out AuthError))
        {
            var replyHandler = new LAContextReplyHandler((success, error) =>
            {
                if (success)
                {
                    onSuccess?.Invoke();
                }
                else
                {
                    onFailure?.Invoke();
                }
            });

            context.EvaluatePolicy(LAPolicy.DeviceOwnerAuthentication, Strings.PleaseAuthenticateToProceed, replyHandler);
        }
    }

    private static LocalAuthType GetLocalAuthType()
    {
        var localAuthContext = new LAContext();
        NSError AuthError;

        if (localAuthContext.CanEvaluatePolicy(LAPolicy.DeviceOwnerAuthentication, out AuthError))
        {
            if (localAuthContext.CanEvaluatePolicy(LAPolicy.DeviceOwnerAuthenticationWithBiometrics, out AuthError))
            {
                if (GetOsMajorVersion() >= 11 && localAuthContext.BiometryType == LABiometryType.TypeFaceId)
                {
                    return LocalAuthType.FaceId;
                }

                return LocalAuthType.TouchId;
            }

            return LocalAuthType.Passcode;
        }

        return LocalAuthType.None;
    }

    private static int GetOsMajorVersion()
    {
        return int.Parse(UIDevice.CurrentDevice.SystemVersion.Split('.')[0]);
    }
}

There are helper methods determining proper label (GetLocalAuthLabelText), icon (GetLocalAuthIcon) and authentication text (GetLocalAuthUnlockText) depending on available authentication type. There is also one liner IsLocalAuthAvailable checking if Local Authentication (face/touch ID or passcode) is available, and Authenticate method that performs authentication, which takes success and failure callbacks as parameters. It can be used in WillEnterForeground method as follows:

public override void WillEnterForeground(UIApplication application)
{
    if (!AppSettings.IsLocalAuthEnabled)
    {
        return;
    }

    LocalAuthHelper.Authenticate(null, // do not do anything on success
    () =>
    {
        // show View Controller that requires authentication
        InvokeOnMainThread(() =>
        {
            var localAuthViewController = new LocalAuthViewController();
            Window.RootViewController.ShowViewController(localAuthViewController, null);
        });
    });
}

We do not have to do anything on success. The popup shown by iOS will disappear and user will be able to use the app. On failed authentication though we should display some kind of shild (e.g., ViewController) that prevent user from using the app until authorization succeed. This is how it looks in Azure App:

Azure App - Unlock with Touch ID

Android

Detecting when app is coming from background in Android is tricky. There is no single method that is invoked only when app is coming back from background. The OnResume method is being called when app is coming back from the background, but it’s also called when you switch from one activity to another. Solution for that is to keep a time stamp with last successful authentication, and update it to DateTime.Now every time when activity is calling OnPause. This happen when app is going to background, but also when app is changing between activities. Thus we cannot simply set flag Background=true when OnPause is called. However, when difference between subsequent OnPause and OnResume is larger than some period of time (e.g., more than a few seconds) we can assume that app went to background. Below code should be implemented in some BaseActivity class that all activities inherit from:

public class BaseActivity
{
  public const int FingerprintAuthTimeoutSeconds = 5;
  public static DateTime LastSuccessfulFingerprintAuth = DateTime.MinValue;
    
  protected override void OnResume()
  {
    base.OnResume();

    if (IsFingerprintAvailable() && LastSuccessfulFingerprintAuth > DateTime.Now.AddSeconds(-FingerprintAuthTimeoutSeconds))
    {
      StartActivity(typeof(FingerprintAuthActivity));
    }
  }

  protected override void OnPause()
  {
    base.OnPause();

    if (IsFingerprintAvailable())
    {
      LastSuccessfulFingerprintAuth = DateTime.Now;
    }
  }
}

The basics of Fingerprint authentication are very well described in Xamarin docs.

Even better reference is a sample app FingerprintGuide from Xamarin.

The main disadvantage of adding fingerprint authentication in Android (over Face/Touch ID in iOS) is requirement to build your own UI and logic for the authentication popup. This includes adding icon, and handling all authentication results. iOS handles incorrect scan, and displays popup again with passcode fallback after too many unsuccessful tries. In Android you have to implement this entire logic by yourself.

Summary

Adding biometrics authentication is useful for apps that hold sensitive data, like banking apps, file managers (Dropbox, OneDrive), or an app that has access to your Azure Resources 🙂

Implementing local authentication in iOS is pretty straightforward, and iOS APIs provide authentication UI for free. In Android however, the APIs are only working with the backend, and UI has to be implemented by you.

Local authentication should be always optional. Some users may not need nor want it. Thus, it should be configurable in the app settings.

Try out biometrics auth in Azure App!

Download on the App Store
Get it on Google Play


Managing multiple accounts in Azure App

One of our top user’s feedback requests was to enable multiple account access without singing out and signing in.

It is now available on latest iOS and Android releases!

Quick overview

Azure App - Multiple Accounts

You can see all your accounts in hamburger menu. You can add new account by tapping ‘Add account’. To remove account you need to simply sign out.

Limitations

First limitation (AKA caveat): when you are adding second live account you, you may run into the following screen during authentication with Active Directory:

Azure App - already signed in

This is Active Directory limitation. Just tap ‘Sign out and sign in with a different account’, and we will sign you in to another account without signing you out from another.

Second limitation: you are not able to sign in into two accounts if they are associated with one email. Yes! One email can be used to sign in to more than 1 account. If you are seeing screen similar like below during authentication this is a case:

Azure App - Multi Accounts One Email

In this situation you need to sign out, and then sign in again choosing another account.

Summary

I personally love this feature as switching between my MSDN account and work account was a pain in the past. Now it’s seamless.

We are still exploring possibility to enable users to sign in into two accounts tied to the same email. We are also looking at improvements around removing and adding accounts.

Do you have feedback? Let us know! You can ping me or our team on twitter. You can also add or vote on existing ideas at our User Voice.

Happy Thanksgiving!


In-memory caching in Xamarin apps

CPU

Recently we added in-memory caching to Azure App. You can try it out now on iOS and Android!

It turns out Mono doesn’t have System.Runtime.Caching namespace, which makes it easy to implement caching for .NET apps. We had to find another way.

Caching libraries for Xamarin

We looked at a few libraries for caching (e.g., MemoryCache and Akavache), but surprisingly none of them manage cache size and memory. They simply add items to Dictionary, and if you add too many you get OutOfMemoryException.

It may not be an issue for many applications, but in Azure App we need to take into account users who has multiple subscriptions with thousands of resources.

BTW: Akavache is a great library. Besides in-memory cache it also supports persistent cache, have clean APIs and a lot of great documentation.

Implementing in-memory cache

After browsing internets and asking people at Xamarin chat we didn’t find anything that would work for us, and we decided to implement in-memory cache by ourselves.

public class InMemoryCache<T> : IInMemoryCache<T>
{
    private const int LimitedCacheThreshold = 1000;

    private class Reference
    {
        private int _hitCount = 0;

        public DateTimeOffset Timestamp
        {
            get;
            private set;
        }

        public T Data
        {
            get;
            private set;
        }

        public void AddRef()
        {
            Interlocked.Increment(ref _hitCount);
        }

        public int ResetRef()
        {
            var count = _hitCount;
            _hitCount = 0;
            return count;
        }

        public static Reference Create(T obj)
        {
            return new Reference()
            {
                Timestamp = DateTimeOffset.Now,
                Data = obj,
            };
        }

        private Reference()
        {
        }
    }

    private readonly ConcurrentDictionary<string, WeakReference<Reference>> _weakCache;
    private readonly ConcurrentDictionary<string, Reference> _limitedCache;
    private readonly ConcurrentDictionary<string, Task<T>> _pendingTasks;

    private InMemoryCache()
    {
        _weakCache = new ConcurrentDictionary<string, WeakReference<Reference>>(StringComparer.Ordinal);
        _limitedCache = new ConcurrentDictionary<string, Reference>(StringComparer.Ordinal);
        _pendingTasks = new ConcurrentDictionary<string, Task<T>>(StringComparer.Ordinal);
    }

    public static IInMemoryCache<T> Create()
    {
        return new InMemoryCache<T>();
    }

    public async Task<T> GetOrAdd(string key, DateTimeOffset expiration, Func<string, Task<T>> addFactory)
    {
        WeakReference<Reference> cachedReference;

        if (_weakCache.TryGetValue(key, out cachedReference))
        {
            Reference cachedValue;
            if (cachedReference.TryGetTarget(out cachedValue) || cachedValue != null)
            {
                if (cachedValue.Timestamp > expiration)
                {
                    cachedValue.AddRef();
                    return cachedValue.Data;
                }
            }
        }

        try
        {
            var actualValue = await _pendingTasks.GetOrAdd(key, addFactory);

            if (_limitedCache.Count > LimitedCacheThreshold)
            {
                var keysToRemove = _limitedCache
                    .Select(item => Tuple.Create(
                        item.Value.ResetRef(),
                        item.Value.Timestamp,
                        item.Key))
                    .ToArray()
                    .OrderBy(item => item.Item1)
                    .ThenBy(item => item.Item2)
                    .Select(item => item.Item3)
                    .Take(LimitedCacheThreshold / 2)
                    .ToArray();

                foreach (var k in keysToRemove)
                {
                    Reference unused;
                    _limitedCache.TryRemove(k, out unused);
                }
            }

            var reference = Reference.Create(actualValue);
            _weakCache[key] = new WeakReference<Reference>(reference);
            _limitedCache[key] = reference;

            return actualValue;
        }
        finally
        {
            Task<T> unused;
            _pendingTasks.TryRemove(key, out unused);
        }
    }
}

We use two layers of caching. First is using WeakReference that leaves memory management to Garbage Collector. As GC is not very predictable and sometimes may unnecessary release some reference, we have second layer of caching. We call it _limitedCache, and it keeps objects in memory until capacity reach 1000 objects. Then we remove half (500), least used objects from dictionary. Because the same objects are being kept in two dictionaries, the WeakReference will never be released as long as object is in _limitedCache. Thus, we always check only if object is present in _weakCache.

There is also third dictionary that keeps track of pending tasks that are responsible for getting data. This prevents us from sending the same requests more than once if object is not in cache yet.

Summary

What is great about building apps with Xamarin is the ability to share code across platforms. When we were implementing cache, we didn’t touch any platform specific code. All work was done in Portable Class Library.

Adding cache to Azure App helped not only to decrease user’s network data usage, but also to improve performance significantly!

If you need in-memory cache for your app, go ahead and use the above code snippet! If you are looking for persistent cache then consider using Akavache.

Are you caching? How? Why? Why not?


Trying iOS 11 with Xamarin

The triathlon season is over. I completed all three, planned races for this year:

  1. Ironman 70.3 Coeur d’Alene
  2. SeaFair Sprint Triathlon (new PR!)
  3. Lake Meridian Olympic Triathlon (new PR!)

I also finished RAMROD (epic Ride Around Mt Rainier in One Day) and Course d’Equipe. The last bike ride for this season is Gran Fondo Whistler in two weeks.

In the meantime…

The Winter is coming!

Apple is cooking for us iOS 11, and I decided to give it a shot! It actually works nice.

  1. Install latest Xcode beta from here
  2. Install latest Xamarin.iOS (all links are here, hint: version is 10.99, not 11 yet)
  3. Set VS for Mac to Xcode-beta (Preferences -> Projects -> SDK Locations -> Apple -> Location)

If you did everything correct you should be able to see new iOS11 simulator:

iOS 11 simulator

I encountered one issue: when deploying to device I got following errors:

Error: unable to find utility “lipo”, not a developer tool or in PATH
Error: Failed to create the a fat library

Solution was to run the following command:

sudo xcode-select --switch /Applications/Xcode-beta.app/Contents/Developer/

Related Xamarin Forums thread.

Summary

So far everything works pretty well. Occasionally when I run VS for Mac it doesn’t detect simulators, but after restart they are back!

Have you tried iOS 11 yet?