The Minimum For Integration Logins

When you want to have your site support login from other services like Google, Facebook, or Microsoft, you use the Owin libraries.  The base MVC template sets all this up for you, but it also has a lot of stuff that your site probably already has, like login and a user database and whatnot.

So, what’s the minimum amount of code you need to add to handle the extra login sources.  You need two things: a class to initialize Owin and an MVC controller to handle the login redirect and the login response from the external site.

This is the class to initialize Owin:

Imports Microsoft.Owin
Imports Microsoft.Owin.Security
Imports Owin.Security.Providers

' Nuget Packages needed:
' Owin
' Microsoft.Owin
' Microsoft.Owin.Security
' Microsoft.Owin.Security.Coookies
' Microsoft.Owin.Host.SystemWeb
' Microsoft.Owin.Security.Facebook (for FB Login)
' Microsoft.Owin.Security.Google (for Google Login)
' Microsoft.Owin.Security.MicrosoftAccount (for MS Login)
' Owin.Security.Providers (for many other Logins)

<Assembly: Microsoft.Owin.OwinStartup(GetType(Startup))>
Public Class Startup
    Public Sub Configuration(app As Owin.IAppBuilder)
        Dim opt As Cookies.CookieAuthenticationOptions

        opt = New Cookies.CookieAuthenticationOptions With {.LoginPath = New PathString("/Account/Login")}

        Owin.CookieAuthenticationExtensions.UseCookieAuthentication(app, opt)
        AppBuilderSecurityExtensions.SetDefaultSignInAsAuthenticationType(app, opt.AuthenticationType)

        ' Google - Signup https://developers.google.com
        Owin.GoogleAuthenticationExtensions.UseGoogleAuthentication(app, New Google.GoogleOAuth2AuthenticationOptions With {.ClientId = "", .ClientSecret = ""})

        ' Facebook - Signup https://developers.facebook.com/
        Owin.FacebookAuthenticationExtensions.UseFacebookAuthentication(app, New Facebook.FacebookAuthenticationOptions With {.AppId = "", .AppSecret = ""})

        ' Microsoft - Signup https://account.live.com/developers/applications/index (RedirectURL in app settings must be http://domain.com/signin-microsoft) 
        Owin.MicrosoftAccountAuthenticationExtensions.UseMicrosoftAccountAuthentication(app, New MicrosoftAccount.MicrosoftAccountAuthenticationOptions With {.ClientId = "", .ClientSecret = ""})

        ' Yahoo - Signup http://developer.yahoo.com (not working; 401 errors) 
        Yahoo.YahooAuthenticationExtensions.UseYahooAuthentication(app, New Yahoo.YahooAuthenticationOptions With {.ConsumerKey = "", .ConsumerSecret = ""})

        ' Flickr - Signup https://www.flickr.com/services/api/keys/
        Flickr.FlickrAuthenticationExtensions.UseFlickrAuthentication(app, New Flickr.FlickrAuthenticationOptions With {.AppKey = "", .AppSecret = ""})

    End Sub

End Class

And this is the controller class to manage the logins:

Imports System.Web.Mvc

Namespace Controllers
    Public Class OwinController
        Inherits Controller

        Function Login(provider As String, returnURL As String) As ActionResult
            Return New ChallengeResult(provider, Url.Action("Callback", "Owin", New With {.ReturnURL = returnURL}))
        End Function

        Function Callback(returnURL As String)
            Dim userID As String
            Dim email As String

            ' capture credentials
            With DirectCast(My.User.CurrentPrincipal.Identity, Security.Claims.ClaimsIdentity)
                userID = .Claims.Where(Function(x) x.Type.EndsWith("/nameidentifier")).DefaultIfEmpty(New Security.Claims.Claim("", "")).First.Value
                email = .Claims.Where(Function(x) x.Type.EndsWith("/emailaddress")).DefaultIfEmpty(New Security.Claims.Claim("", "")).First.Value
            End With

            Debug.WriteLine(My.User.CurrentPrincipal.Identity.Name)

            Return New RedirectResult(returnURL)

        End Function

        Private Class ChallengeResult
            Inherits HttpUnauthorizedResult

            Public Property LoginProvider As String
            Public Property RedirectURL As String

            Public Sub New(provider As String, url As String)
                Me.LoginProvider = provider
                Me.RedirectURL = url
            End Sub

            Public Overrides Sub ExecuteResult(context As ControllerContext)
                Dim prop As New Microsoft.Owin.Security.AuthenticationProperties With {.RedirectUri = Me.RedirectURL}
                context.HttpContext.GetOwinContext.Authentication.Challenge(prop, Me.LoginProvider)
            End Sub

        End Class

    End Class

End Namespace

The Callback method in the OwinController is where you would look up the user by their provider/nameidentifier combination or store that info in an existing logged-in user profile or create a new user profile for the new login.

When you want to offer external logins, you call the Owin controller:

@Html.ActionLink("Google Login", "Login", "Owin", New With {.Provider = "Google", .ReturnURL = Url.Action("Secure")}, Nothing)
<br />
@Html.ActionLink("Facebook Login", "Login", "Owin", New With {.Provider = "Facebook", .ReturnURL = Url.Action("Secure")}, Nothing)
<br />
@Html.ActionLink("Yahoo Login", "Login", "Owin", New With {.Provider = "Yahoo", .ReturnURL = Url.Action("Secure")}, Nothing)
<br />
@Html.ActionLink("Flickr Login", "Login", "Owin", New With {.Provider = "Flickr", .ReturnURL = Url.Action("Secure")}, Nothing)
<br />
@Html.ActionLink("Microsoft Login", "Login", "Owin", New With {.Provider = "Microsoft", .ReturnURL = Url.Action("Secure")}, Nothing)

This assumes there is an action named “Secure” that you want to come back to.

Outlook Rules Utility

Way back in 2013, I was getting annoyed with the rules manager in Outlook.  See, I have many rules.  I have one folder for every person in my inbox.  The rules move their messages to their respective folders.  So at work, I have 67 rules.  At home, I have 267 rules.  I like to keep my rules sorted, so I can find one quickly if I need to.  So if I add a rule for Zillow.com, I have to click the “move down” arrow hundreds of times to get to the bottom of the list.  There has to be a better way.

I focused on the rules export, which creates a binary file of all your rules.  I just had to reverse-engineer the file structure and I could create an app to modify the order of the rules.  Over a span of time, I figured out what I needed and the results were somewhat workable.  I uploaded the source code to CodePlex and didn’t really use it much after that.

Two and a half years later, I was annoyed at my rules again and fired up this code to fix my rules.  I discovered the code didn’t work like I expected, so I set to work fixing the problems.  Once it was working properly, I decided that the CodePlex project deserved to have a binary for download.  It wasn’t doing a lot of good to people who weren’t programmers and who could compile the code on their own.

After that update, I did a quick search to see if anything new had been learned about the Outlook rules file since I last worked on it.  Nothing.  But what I did stumble on was a post asking how to parse the rules file.  In the post, the author explained where the rules file was stored in the mail store file.  Taking that information, I was able to add the ability for my utility to read and write the rules directly from Outlook.  No more export/import hassle!  That is a huge step forward.

This unexpected progress has inspired me to work a little harder on the project.  There are random bits of code and comments that handle the different criteria and the different actions.  These need to be clarified and once that is done, this utility could be an entire replacement for the Rules Manager in Outlook.

The project is at http://rwzreader.codeplex.com.

GrooveManager 1.0 Release

I’ve been posting for a little while on Groove Music’s database.  While investigating, I wrote up some test utilities to read and write to the Groove Music database.  These utilities have been combined and simplified into a utility called GrooveManager.

At this point, the utility is extremely limited.  It will only edit Artist names and the Artist image.  And the Artist image feature itself is limited in that you can’t use external images.  It has to be an artist with a profile on Xbox Music.  That limitation aside, there are still times when you have a legitimate need to use an artist that isn’t spelled the same as you have it in your collection.  For example, Sergei Rachmaninoff seems to have many different spellings and variations.

The project page is at https://groovemanager.codeplex.com and has simple documentation on the website.

Groove Metadata (database)

Here’s some more info on the structure of the Groove Music database.

Schema

As mentioned the database is in ESE format.  The tables within are pretty much just like any other database.  And in Groove Music’s case, you have the expected elements of Artist, Album, Tracks, and Genre.  Most of the fields are what you would expect them to be.  Here’s a brief analysis on some things you might be curious about.

tblPerson

This table holds the Artist info, but it’s more than just the band name.  It also includes credits in each song’s ID3 tag, like composer, artist, albumartist, etc.  This means you’re going to have a lot more “Persons” than bands in your library.  Especially when dealing with compilation albums.  Other fields:

tblAudioAlbum

This table is the list of albums in the library.

tblTrack

This table lists the songs.  There’s lots of interesting info in here that needs more exploration.  I have three albums in the cloud.  Two I purchased back from the Zune days, and one I just uploaded to OneDrive from my hard drive.  I’ll reference these in the interesting fields.

  • CloudCollectionContentId – This seems to be a guid to use in a link to an online file.  This is populated for both my purchased and uploaded albums
  • BlockFromCloud – Always true
  • ActionableMediaId – I assume this is a Guid that points to a matched online track. It’s blank on my more obscure albums.
  • InCloudCollection – True for the three albums I have in OneDrive and Store-purchased
  • CatalogId – The same as ActionableMediaId, but there are more CatalogIds than ActionableMediaIds.  Maybe ActionableMediaId means it can be used online instead of just having information?
  • AlbumImageId – A Guid that is used to get the album art from web services
  • AlbumImageSource – A number that likely represents the URL to get the image from (zuneimages, music.microsoft.com, or musicimages.xbox.com).
  • UniqueFileId – These values are similar to what Windows Media Player used to store in the ID3 tag when it would apply its metadata from its store.

So still, plenty to learn about this database.

Groove Metadata (album art)

As a previous post mentioned, the metadata for Groove is in an ESENT database.  One of the big questions is where does it get its artwork from?

Album Artwork

Groove will get its album artwork from one of two places: from the file metadata or from xbox catalog web services at http://musicimage.xboxlive.com  (probably former Zune services).  It does not use folder.jpg, nor does it use ZuneArt_{guid}.jpg.  It will only access the web services if you have the Media Info option set to automatically retrieve missing metadata.

In order to get good album artwork, your metadata needs to be pristine and has to match what’s in the xbox metadata services.  That’s unreliable at best.  The best solution is to embed your artwork in your media files.

If you want to wipe out your library and rebuild from scratch, you can rename the randomly-(or not)-named folder in:

%userprofile%\AppData\Local\Packages\Microsoft.ZuneMusic_8wekyb3d8bbwe\LocalState\Database

Why wipe out the library?  If you have a massive amount of artwork that originally was downloaded from the xbox services and now you have all that artwork in your media files, it’s faster to let Groove rescan your library than to update each album’s artwork.

As Groove is collecting the album art, it keeps a cache of the images in:

%userprofile%\AppData\Local\Packages\Microsoft.ZuneMusic_8wekyb3d8bbwe\LocalState\ImageStore

It doesn’t seem to matter if the image comes from the file or from the web services.

Artist Images

How, when and where does Groove get its artist info?

Although the ID3 specification does provide a way to embed an artist image, there isn’t currently a common utility that lets you do it.  Doesn’t matter anyway since Groove doesn’t read the metadata fro artist images.

Like the album art cache, artists are stored in:

%userprofile%\AppData\Local\Packages\Microsoft.ZuneMusic_8wekyb3d8bbwe\LocalState\ImageCache\20

And the way Groove gets the artist images is by searching the music.xbox.com site based on your metadata.  If there’s no match,m or if music.xbox.com doesn’t have that artist in the marketplace, that’s about where it ends.  There doesn’t appear to be a way to override this yet.

Editing SharePoint Pages Using Visual Studio And WebDAV

I blog in a few different places.  I have my personal blog, my professional blog, and I maintain a blog at work to inform and educate co-workers.  At work, the blog is hosted on the company’s SharePoint server, which is fine.  I am still able to use Windows Live Writer and with it, the Insert Code plug-in.  My other two blogs use WordPress.

The Insert Code plug-in is invaluable to me because it does nice color coding of the text.  As part of that feature, it inserts a CSS style block into your post.  SharePoint doesn’t play well with this.  It tries to, but fails.  The intent is good.  SharePoint wraps your whole post in a div and gives it a class with a random name, then it rewrites the CSS styles so the classes will be scoped to only that containing div class.  Pretty smart way of encapsulating the styles.

Unfortunately, it fails on two points.  First, the containing div’s class is not like class=”123456789abcd”, it always precedes the class name with “ExternalClass”, so you get class=”ExternalClass123456789abcd”.  The rewritten CSS does not have any mention of “ExternalClass”.

The second mistake is in the rewritten CSS.  Your post will have a style block rewritten similar to:

<style>
123456789abcd h1 {color: red;}
123456789abcd h3 {color: green;}
123456789abcd .bold {font-weight: bold;}
</style>

Do you see the problem?  The class is 123456789abcd on the div (actually ExternalClass123456789abcd), but the stylesheet doesn’t scope it to that class.  Those are html tags it’s defining.  The stylesheet is looking for html tags of <123456789abcd>.

So, what can you do about this?  My solution was to put the stylesheet right in the template page so all the posts will be able to use those classes – that’s why I created this post.  The problem is, I couldn’t find anything in the SharePoint control panel to add a custom stylesheet (unlike WordPress, right?).  There was an option to edit the page using SharePoint Designer, so I installed Designer, only to find out the administrator didn’t allow editing using Designer. 

Other places on the Internet suggested adding a Content Editor web part and put the style sheet in there.  I tried it half-heartedly and gave up because it seemed way too “hack-y”.  But while doing so, it reminded me of something I used to know about SharePoint, that you could browse the site’s files using WebDav (assuming you had permissions).

So, what I did was map a network drive (a command found in many places in Windows Explorer) and gave it the URL of my SharePoint site.  Right away, I got an Explorer window with the template files.  I edited Default.aspx and Post.aspx and added my stylesheet.  The formatting was immediately applied.  Then I edited all my previous posts and removed my inline style code blocks to save space and reduce complexity.  Everything works now.

Visual Studio 2015 + Silverlight In VB.NET

At work, I was the first to upgrade my machine to Windows 10 and Visual Studio 2015.  Opening our application’s solution, I started resolving little errors here and there.  One problem was not so simple and had a very obscure solution.

We have a small collection of Silverlight apps written in VB.NET.  With the upgrade to Visual Studio 2015, they would not compile.  The errors returned were:

Microsoft (R) Visual Basic Compiler version 1.0.0.50618
Copyright (C) Microsoft Corporation. All rights reserved.

vbc : error BC30002: Type ‘Global.Microsoft.VisualBasic.ApplicationServices.ApplicationBase’ is not defined.
vbc : error BC30002: Type ‘Global.Microsoft.VisualBasic.Devices.Computer’ is not defined.
vbc : error BC30002: Type ‘Global.Microsoft.VisualBasic.MyServices.Internal.ContextValue’ is not defined.
vbc : error BC30002: Type ‘Global.Microsoft.VisualBasic.ApplicationServices.User’ is not defined.
vbc : error BC30002: Type ‘Global.System.ComponentModel.Design.HelpKeyword’ is not defined.

In the course of troubleshooting, I excluded everything from the project except the AssemblyInfo file and I still got the errors.  So I created a new Silverlight project, which built fine, and compared the Detailed compiler output.  The compiler line in the 2015-created project had a slightly different parameter (difference highlighted):

/define:"CONFIG=\"Debug\",DEBUG=-1,TRACE=-1,_MyType=\"Empty\",PLATFORM=\"AnyCPU\",SILVERLIGHT=1"

This clicked with me from my experience with building Silverlight apps for the Zune, where you had to exclude the My namespace that is created with VB.NET applications.  Look at the assemblies being referenced.  ApplicationServices.ApplicationBase (My.Application), Devices.Computer (My.Computer), ApplicationServices.User (My.User).

So, how do you get this extra compile setting?  It’s in the .vbproj – there’s no UI for it.  You’ll have to edit the vbproj file directly.

Unload the Silverlight project, right-click it, and choose Edit .vbproj.  Look for the node <NoStdLib>true</NoStdLib> (there will be one per project configuration) and after it, add: <MyType>Empty</MyType>.  This node is included with new Silverlight projects in VS2015.

Save and reload the project and the MyType flag will be included in the vbc command line, which will allow the Silverlight application to compile.

Of course, a lot of people will probably create a new Silverlight project and move all their code into it and be just as successful.  But, if you don’t want that hassle, or if you just want to know why (as I did), there’s the solution.

Windows 10 Groove Music – Zune On

Overview

With the release of Windows 10 comes a new music application, Groove Music.  Groove Music has Zune DNA, except that it loses any Windows Media Player (WMP) or Zune syncing capability.  The assumption is that the mobile phone is the new MP3 player and file copy is the preferred method of syncing.  For better or worse, this is the new normal.

Groove is much closer to the aesthetics of Zune than of WMP, and aside from the lost syncing capability and the toned-down Now Playing screen, it’s a reasonable Zune replacement – as a music player.  Syncing, well… not as much.  You have your usual views: Artist, Album, Song, Playlists, plus Albums for an Artist.  Genre view is missing.  Typing will expand the hamburger menu and put the text in the search box, proving immediate search.  Of course you have the Marketplace, to purchase and download more content.

Technical Details

Groove is a successor to Zune, although the outward branding does not hint at it.  The code library is called ZuneMusic and is found at %userprofile%\AppData\Local\Packages\Microsoft.ZuneMusic_8wekyb3d8bbwe.  In the subfolder LocalState you will find plenty of runtime details.  LocalState has a folder for the database, which is in ESENT format.  There are the ImageCache and imageStore folders that hold album artwork and artists photos from the Zune web services.

As far as the database is concerned, it seems to be similar if not exact to the old Zune database, which was in SQL Server Compact format.  The most common tables would be: tblAudioAlbum, tblPerson, tblGenre, and tblTrack to hold the music metadata and tblFolder and tblFile to hold the physical file references.

There are utilities and libraries to work with ESENT databases.  One is called ESENT Workbench.  If you do want to play around with the database, you may need to do a repair on it because it may not have shut down cleanly.  You can run the command “esentutl.exe /p EntClientDb.edb” to clean up the files for reading.

The Groove Music app uses another couple of packages extensively: Microsoft.Windows.CloudExperienceHost and Microsoft.Windows.ContentDeliveryManager, but probably not for primary functions.  The majority of data is likely pulled from the Zune web services, since the entries in the matadata database tables have references to GUIDs that, when used with the web services, retrieve the proper artist or album info.

The database for Groove Music appears to sync between computers, which makes a lot of sense for cloud-based music, but may get hairy when different machines have different local files.

Extension Ideas

What can be done with having access to the music database?  My impetus for this research was trying to change the Now playing slideshow to use all the artist pictures like Zune does, instead of a single album picture.  I haven’t gotten that far yet.  But some ideas for apps would be:

  • Statistic app showing most played artists, albums, songs, genres
  • Smart playlist generator based on statistics
  • Statistics on files: sizes, bitrates, dates, and something intriguing called FingerprintData
  • Utility to clean, purge, delete, export library
  • Post Now Playing, Recently Played to social media
  • Create a smart sync utility that utilizes the library’s metadata with file copy

Azure Aftermath

After the mess with the Azure SDK reserving ports that conflicted with my development environment, and the fact that the SDK didn’t do anything for the automated publishing I was looking for, I decided to uninstall it.  Well, that wasn’t a good idea.  You could say it was “Azure disaster.”

First off, the uninstall of the multiple elements of the Azure SDK did not remove the NETSH entries for the Azure Storage Emulator. so those had to be manually removed. Then, I reopened VS 2013 and suddenly none of my web projects open, saying they are all incompatible with this version of Visual Studio.  I tried to create a new web application and got the useful error:

image

So now I can’t open or create any web applications in Visual Studio.  I decide I should reinstall the Azure SDK to put back what was taken away, although in my mind I realize this is a ridiculous concept.

I download the SDK and run it.  I then get this error from the installer:

image

Absolutely incredible.  This is like dealing with anti-psychotic medication.  “Whoa whoa whoa, you just can’t uninstall Pericyazine, that can cause a psychotic episode.”

Next thing to do is a Repair of the Visual Studio installation, which is going to be at least an hour…  And that has fixed the web project issue, so at least uninstalling the Azure SDK is recoverable.  You just need to make sure you manually get rid of the ACL entries with these statements (at an Administrator command prompt):

netsh http delete urlacl url=http://127.0.0.1:10000/
netsh http delete urlacl url=http://127.0.0.1:10001/
netsh http delete urlacl url=http://127.0.0.1:10002/

Yes, I do believe when Windows 10 and VS 2015 come out, this work machine is due for a reformat.

IISExpress – Error description: Access is denied. (0x80070005)

Today, I spent my entire work day trying to find the solution to this error.

When I would launch a web project from Visual Studio, IIS Express would fail to load.  The error reported was:

Failed to register URL "http://localhost:10000/" for site "Portal3" application "/". Error description: Access is denied. (0x80070005)

I started down the obvious track of checking permissions.  IISExpress would operate just fine when run as Administrator, but not as my normal user.  I was an administrator on the local machine, so that didn’t make any sense.

The only thing I had done that morning was use Disk Cleanup to delete all my temp files and archived error reports.  I checked %temp%\iisexpress.  I checked my *.config files in Documents\IISExpress over and over.  I restored configs from backups.  I read many potential solutions, some as crazy as not being able to use a user name with “bg” in it.

I tried an older iteration of the web application.  It worked.  I tried other web projects in the solution.  They worked.  At this point, I was the only developer in my team who had a single web project that wouldn’t launch.

Then, late as usual, I finally remembered the Windows Event Viewer.  IISExpress was logging these two events:

Event ID: 2269: The worker process for app pool ‘Clr4IntegratedAppPool’, PID=’7536′, failed to initialize the http.sys communication when asked to start processing http requests and therefore will be considered ill by W3SVC and terminated.  The data field contains the error number.

EventID 2276: The worker process failed to initialize correctly and therefore could not be started.  The data is the error.

That lead me down a totally different troubleshooting path.  While doing so, I set up a new website in IIS on port 8085.  Oddly, my new websites and application pools didn’t refresh after being created, which led me to think there was even more wrong with my workstation.  An IISRESET resolved that problem and I was able to browse to the web project using IIS.

So IIS would work on port 8085 but IISExpress would not work on port 10000.  And IIS would work for any other project on seemingly any other port. 

In a comment on one of the web posts that I read, I saw this command:

netsh http add urlacl url=http://localhost:10000/ user=everyone

I’d only ever used NETSH maybe once or twice for some obscure reason, but the inclusion of “ACL” in the command was encouraging.  Amazingly, the command worked!  Although I was happy, I was also disappointed that I wouldn’t be able to see what the value was previously so I could find out the root cause.

So I ran:

netsh http show urlacl

And that displayed a bunch of entries that included these as well (User and SDDL changed):

Reserved URL            : http://127.0.0.1:10000/
    User: Me 
        Listen: Yes
        Delegate: No
        SDDL: D:(A;;GX;;;S-1-5-21-)

Reserved URL            : http://127.0.0.1:10001/
    User: Me
        Listen: Yes
        Delegate: No
        SDDL: D:(A;;GX;;;S-1-5-21-

Reserved URL            : http://127.0.0.1:10002/
    User: Me
        Listen: Yes
        Delegate: No
        SDDL: D:(A;;GX;;;S-1-5-21-)

Where did these come from?  I ran netsh http delete urlacl url=http://localhost:10000/ and removed the entry and confirmed IISExpress stopped working.  Then I ran netsh http delete urlacl url=http://127.0.0.1:10000/ and IISExpress started working again.

Something had added ACL entries for ports 10000-10002 that were conflicting with my web project, which was trying to run on port 10000 using IISExpress.  How did they get there?  I looked in Add/Remove Programs to get a clue as to what could have added these entries.  Who’s to blame?  The Azure SDK.  Azure Storage Emulator uses ports 10000-10002 and creates reservations for them.  This was installed during my attempts at getting command-line Web Publishing to work.

I never would have known anything about this unless I had read the Azure SDK documentation.  The error message I was given said nothing about port conflicts.  nothing led me down that path at all.  And it’s entirely possible no one else would have this problem unless they were running IISExpress on port 10000.

But the important takeaway from this is that NETSH will allow you to create reservations for port numbers that may conflict with other applications.