Our new book and my personal journey toward REST and Hypermedia

It’s hard to believe, but it has been 6 months since our new book, “Designing Evolvable Web APIs with ASP.NET” shipped! So far the feedback that we’ve received has been really positive and we’re excited to see the momentum.

book

In the first part of the book we focus on educating about the fundamentals of the web architecture and HTTP, and APIs. Next we focus on the designing of a hypermedia API, and a TDD/BDD driven implementation of a CollectionJson API using ASP.NET Web API. We also give you several coding patterns that you can use to get started building hypermedia APIs.

In the second part of the book, we focus on ASP.NET Web API itself and the underlying architecture. We do this in the context of the first part, with the explicit goal of helping you further toward building out an evolvable API. We delve deeply into the nuts and bolts of Web API including hosting, routing and model binding. We cover comprehensively how to secure your API, and the security internals. And we cover other topics like TDD and using Inversion of Control.

You can buy it online now here. We also have a Github repo with all the code, which includes a working hypermedia API.

The people that made this book a reality.

This book has been a collaborative effort with four other amazing folks that I’ve had the privilege to work with. It combines the knowledge of the Web API team itself as well as key advisors who were with us every step of the way as we built it. It was a collaborative effort across 4 time zones and countries:

  • Howard Dierking, my co-conspirator on the Web API team. An amazing individual who is helping educate the world on REST and hypermedia through his Pluralsight work.
  • Darrel Miller, a long time proponent of REST and hypermedia in the .NET world before many of us at Microsoft had a clue of what that means. Darrel has been building real hypermedia systems for a long time. Darrel was one of our first handfuls of advisors on Web API.
  • Pedro Felix, an expert on Web API security and the ASP.NET Web API architecture. His comprehensive contribution in the book on the security aspects of Web API is unparalleled in the .NET world.
  • Pablo Cibraro, former CTO of Tellago and a consultant who has implemented many API solutions. Pablo is an expert in Agile and TDD, and he wrote some deep chapters testability and IOC.

We also had a fantastic set of reviewers and advisors.

Why write this book?

This book is part of a very personal journey, a  journey that started several years ago when I joined the WCF team at Microsoft. At that time I had the pop culture understanding of REST, that is a lightweight API exposed over HTTP that doesn’t use SOAP. I joined the team to help build a better experience in the platform for these types of APIs. Little did I know the journey that awaited me, as I would delve deeper and deeper into the REST community.

It was a journey of learning.  I was fortunate to have wonderful teachers who had already been treading the path, chief of which (ordered last name first) were: Jan Algermissen, Mike Amundsen Alan Dean, Mike Kelly, Sebastien Lambla, Darrel Miller, Henrik Nielsen, Ian Robinson, Aaron Skonnard, and Jim Webber. I am deeply indebted for their help. The is book was built really on the shoulders of these giants!

During that period, I learned about HTTP and REST, that REST is much more than building an RPC JSON API, and about how to make APIs more robust. This included learning about fundamental aspects of the web architecture like caching, ETAGS and content negotiation. The learning also included the hidden jewel of REST APIs, hypermedia, and new media types like HAL, CollectionJson and Siren that were designed specifically for hypermedia-based systems.

It was clear looking at our existing Microsoft frameworks that they did need meet the requirements for building APIs that really leveraged HTTP and took advantage of the web architecture. Looking within the .NET OSS community, solutions like Open Rasta, however were explicitly designed with this in mind. There were also a ton of other options outside the .NET world in Java, Ruby, Python and PHP, and more recently in node.js.

After soaking this all in, my team at Microsoft, and a team of fantastic advisors from the community, worked together to create a new framework that was designed from the get-go to fully embrace HTTP, to enable, but not force building a RESTful system.

As part of this we had an explicit goal from day one to ensure this framework would also enable building a fully hypermedia based system. Another explicit goal was to make the framework easier to test, and to allow it to work well with agile development. This framework ultimately became what you now know as ASP.NET Web API.

Although ASP.NET Web API had the foundations in place to enable such systems, you have work to do. I am not going to say that it naturally leads you to build a hypermedia system. This was not an accident. We deliberately did not want to force people down a specific path, we wanted to make sure though if you wanted to build such a system, it wouldn’t fight you.

We saw a lot of folks jump on this, even internally, folks like the Office Lync Team that built probably one of the largest pure hypermedia APIs in existence using Web API.  Many of these early adopters had the advantage of working directly with our team. They worked with me, and Henrik Nielsen (My Web API architect, and one of the creators of HTTP) to help guide them down the path. We did a lot of educating on HTTP, API fundamentals and what hypermedia is and how to build systems that support it.

On the outside we also saw a lot of energy. Folks like our advisory board jumped in and started doing the same in the wild. They built real systems, and they guided customers.

All of this work (including shipping Web API) definitely succeeded in help raising the bar around modern API development and hypermedia in the industry in general. More and more folks including some of the largest companies started coming out of the woodwork and saying, we want to build these types of systems.

However, many folks, in particular in the .NET community, have not crossed this chasm yet and do not understand how or why to build such systems. In particular building a hypermedia-based system is a sticking point. Many are scared by the term and don’t understand what it is. Others dismiss it as purely academic. Even those that do grasp the concepts often do not know where to start to implement them. This last point makes total sense, taking into consideration the points I mentioned earlier that ASP.NET Web API does not lead you toward building such APIs, it enables it.

With this book we wanted to change that. We wanted to bring these API techniques to the masses. We wanted to show you why and how to actually build these APIs with ASP.NET Web API. We wanted to show you how to make the underlying architecture of the framework to work with you in achieving that goal.

I believe we’ve done that. We look forward to your experiences with the book as you embark on API development with ASP.NET.

Posted in ASP.NET, ASP.NET Web API, Hypermedia, patterns | 2 Comments

Quick easy steps to grab your build artifacts from Visual Studio Online

If you haven’t checked out Visual Studio Online yet, you are missing a lot! I’m in the midst of producing a video series on how to use Visual Studio Online for WintellectNOW. Don’t have a WintellectNOW account? No problem. Enter my promo code PETERSEN-14 and get 2 weeks of unlimited free access.

If you are using Visual Studio Online and have begun to use the hosted build controller, you may be wondering how to grab your build artifacts. Deploying to targets such as an Azure Website is simple. However, there are times when all you want are the raw build artififacts (dll’s, exe’s, html, css and js files, etc). How do you get those?

As it turns out, it is pretty simple!

One of nice new features with Visual Studio Online is the new REST Api.

I’ll leave it to you to sort through the Api reference. The one I want to key on is the Api call to get the list of builds:

GET: https://{account}.visualstudio.com/defaultcollection/_apis/build/builds?api-version=1.0-preview.1

The following is an example


{  
   "value":[  
      {  
         "uri":"vstfs:///Build/Build/36",
         "id":36,
         "buildNumber":"Test-Build-One_20140902.1",
         "url":"https://{account}.visualstudio.com/DefaultCollection/_apis/build/Builds/36",
         "startTime":"2014-09-02T12:40:31.583Z",
         "finishTime":"2014-09-02T12:42:08.483Z",
         "reason":"manual",
         "status":"succeeded",
         "dropLocation":"#/119161/drop",
         "drop":{  
            "location":"#/119161/drop",
            "type":"container",
            "url":"https://{account}.visualstudio.com/DefaultCollection/_apis/resources/Containers/119161/drop",
            "downloadUrl":"https://{account}.visualstudio.com/DefaultCollection/_apis/resources/Containers/119161/drop?api-version=1.0&$format=zip&downloadFileName=Test-Build-One_20140902.1_drop"
         },
         "log":{  
            "type":"container",
            "url":"https://{account}.visualstudio.com/DefaultCollection/_apis/resources/Containers/119161/logs",
            "downloadUrl":"https://{account}.visualstudio.com/DefaultCollection/_apis/resources/Containers/119161/logs?api-version=1.0&$format=zip&downloadFileName=Test-Build-One_20140902.1_logs"
         },
         "sourceGetVersion":"LG:(no branch):b792b3c303982b6be3e6105c3e587307fd35381a",
         "lastChangedBy":{  
            "id":"f21f35aa-5c28-462a-91cb-e077e30e0dbd",
            "displayName":"Elastic Build ({account})",
            "uniqueName":"LOCAL AUTHORITY\\Elastic Build ({account})",
            "url":"https://{account}.vssps.visualstudio.com/_apis/Identities/f21f35aa-5c28-462a-91cb-e077e30e0dbd",
            "imageUrl":"https://{account}.visualstudio.com/DefaultCollection/_api/_common/identityImage?id=f21f35aa-5c28-462a-91cb-e077e30e0dbd"
         },
         "retainIndefinitely":false,
         "hasDiagnostics":true,
         "definition":{  
            "definitionType":"xaml",
            "id":17,
            "name":"Test-Build-One",
            "url":"https://{account}.visualstudio.com/DefaultCollection/_apis/build/Definitions/17"
         },
         "queue":{  
            "queueType":"buildController",
            "id":2477,
            "name":"Hosted Build Controller",
            "url":"https://{account}.visualstudio.com/DefaultCollection/_apis/build/Queues/2477"
         },
         "requests":[  
            {  
               "id":36,
               "url":"https://{account}.visualstudio.com/DefaultCollection/_apis/build/Requests/36",
               "requestedFor":{  
                  "id":"874bcfb5-80fe-4a9f-ac62-25bb3487769f",
                  "displayName":"John V. Petersen",
                  "uniqueName":"{account}@gmail.com",
                  "url":"https://{account}.vssps.visualstudio.com/_apis/Identities/874bcfb5-80fe-4a9f-ac62-25bb3487769f",
                  "imageUrl":"https://{account}.visualstudio.com/DefaultCollection/_api/_common/identityImage?id=874bcfb5-80fe-4a9f-ac62-25bb3487769f"
               }
            }
         ]
      },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  },
      {  }
   ],
   "count":15
}

The item you want to pay attention to is the drop node’s downloadUrl Property:

"drop":{  
   "location":"#/119161/drop",
   "type":"container",
   "url":"https://{account}.visualstudio.com/DefaultCollection/_apis/resources/Containers/119161/drop",
   "downloadUrl":"https://{account}.visualstudio.com/DefaultCollection/_apis/resources/Containers/119161/drop?api-
version=1.0&$format=zip&downloadFileName=Test-Build-One_20140902.1_drop"
}

If you are already authenticated, entering the downloadUrl into a browser will result in a downloaded zip file that contains your deployment artifacts.

There are alternative ways to authorize – which you can find documented here.

That’s it – Super easy to grab your built application artifacts!

Posted in Uncategorized, Visual Studio Online | Leave a comment

Puppet: Getting Started On Windows

Now that we’ve talked a little about Puppet. Let’s see how easy it is to get started.

Install Puppet

PuppetLet’s get Puppet Installed. There are two ways to do that:

  1. With Chocolatey: Open an administrative/elevated command shell and type:
    choco install puppet
    
  2. Download and install Puppet manually – http://puppetlabs.com/misc/download-options

Run Puppet

  • Let’s make pasting into a console window work with Control + V (like it should):
    choco install wincommandpaste
    
  • If you have a cmd.exe command shell open, (and chocolatey installed) type:
    RefreshEnv
  • The previous command will refresh your environment variables, ala Chocolatey v0.9.8.24+. If you were running PowerShell, there isn’t yet a refreshenv for you (one is coming though!).
  • If you are not able to use RefreshEnv (or ‘where puppet’ evaluates to not found), you need to restart your CLI (command line interface) session or open an administrative/elevated command prompt (because you installed manually).
  • Now let’s find out about the users on the system. Type:
    puppet resource user
    
  • Output should look similar to a few of these:
    user { 'Administrator':
      ensure  => 'present',
      comment => 'Built-in account for administering the computer/domain',
      groups  => ['Administrators'],
      uid     => 'S-1-5-21-some-numbers-yo-500',
    }
    
    
    
  • Let’s create a user:
    puppet apply -e "user {'bobbytables_123': ensure => present, groups => ['Users'], }"
    
  • Relevant output should look like:
    Notice: /Stage[main]/Main/User[bobbytables_123]/ensure: created
    
    
    
  • Run the ‘puppet resource user’ command again. Note the user we created is there!
  • Let’s clean up after ourselves and remove that user we just created:
    puppet apply -e "user {'bobbytables_123': ensure => absent, }"
    
  • Relevant output should look like:
    Notice: /Stage[main]/Main/User[bobbytables_123]/ensure: removed
    
    
  • Run the ‘puppet resource user’ command one last time. Note we just removed a user!

Conclusion

You just did some configuration management /system administration. Welcome to the new world of awesome! Puppet is super easy to get started with. This is a taste so you can start seeing the power of automation and where you can go with it. We haven’t talked about resources, manifests (scripts), best practices and all of that yet.

Next we are going to start to get into more extensive things with Puppet. Next time we’ll walk through getting a Vagrant environment up and running. That way we can do some crazier stuff and when we are done, we can just clean it up quickly.

Posted in chocolatey, howto, puppet | Tagged , | 1 Comment

Puppet: Making Windows Awesome Since 2011

imagePuppet was one of the first configuration management (CM) tools to support Windows, way back in 2011. It has the heaviest investment on Windows infrastructure with 1/3 of the platform client development staff being Windows folks.  It appears that Microsoft believed an end state configuration tool like Puppet was the way forward, so much so that they cloned Puppet’s DSL (domain-specific language) in many ways and are calling it PowerShell DSC.

Puppet Labs is pushing the envelope on Windows. Here are several things to note:

It can be overwhelming learning a new tool like Puppet at first, but Puppet Labs has some resources to help you on that path. Take a look at the Learning VM, which has a quest-based learning tool. For real-time questions, feel free to drop onto #puppet on freenode.net (yes, some folks still use IRC) with questions, and #puppet-dev with thoughts/feedback on the language itself. You can subscribe to puppet-users / puppet-dev mailing lists. There is also ask.puppetlabs.com for questions and Server Fault if you want to go to a Stack Exchange site. There are books written on learning Puppet. There are even Puppet User Groups (PUGs) and other community resources!

Puppet does take some time to learn, but with anything you need to learn, you need to weigh the benefits versus the ramp up time. I learned NHibernate once, it had a very high ramp time back then but was the only game on the street. Puppet’s ramp up time is considerably less than that. The advantage is that you are learning a DSL, and it can apply to multiple platforms (Linux, Windows, OS X, etc.) with the same Puppet resource constructs.

As you learn Puppet you may wonder why it has a DSL instead of just leveraging the language of Ruby (or maybe this is one of those things that keeps you up wondering at night). I like the DSL over a small layer on top of Ruby. It allows the Puppet language to be portable and go more places. It makes you think about the end state of what you want to achieve in a declarative sense instead of in an imperative sense.

You may also find that right now Puppet doesn’t run manifests (scripts) in order of the way resources are specified. This is the number one learning point for most folks. As a long time consternation of some folks about Puppet, manifest ordering was not possible in the past. In fact it might be why some other CMs exist! As of 3.3.0, Puppet can do manifest ordering, and it will be the default in Puppet 4. http://puppetlabs.com/blog/introducing-manifest-ordered-resources

You may have caught earlier that I mentioned PowerShell DSC. But what about DSC? Shouldn’t that be what Windows users want to choose? Other CMs are integrating with DSC, will Puppet follow suit and integrate with DSC? The biggest concern that I have with DSC is it’s lack of visibility in fine-grained reporting of changes (which Puppet has). The other is that it is a very young Microsoft product (pre version 3, you know what they say :) ). I tried getting it working in December and ran into some issues. I’m hoping that newer releases are there that actually work, it does have some promising capabilities, it just doesn’t quite come up to the standard of something that should be used in production. In contrast Puppet is almost a ten year old language with an active community! It’s very stable, and when trusting your business to configuration management, you want something that has been around awhile and has been proven. Give DSC another couple of releases and you might see more folks integrating with it. That said there may be a future with DSC integration. Portability and fine-grained reporting of configuration changes are reasons to take a closer look at Puppet on Windows.

Yes, Puppet on Windows is here to stay and it’s continually getting better folks.

Posted in puppet | Tagged , | 1 Comment

blogger { ‘Rob Reynolds’: ensure => present }

Howdy. My name is Rob. I currently have a long time blog at http://ferventcoder.com where I talk about all kinds of things including tools and code that seems useful. I used to blog with Devlicio.us (the sister site) where I showed the code instead of just talking about it. I plan to continue that tradition here. I appreciate the opportunity to share perspectives and thoughts with the community as part of the CodeBetter family.

Thanks for having me and expect to hear more about Chocolatey and Puppet.

Posted in chocolatey, puppet | Tagged , | 1 Comment