Visual merge representation with SemanticMerge

Since my last blog post about the SemanticMerge software (actually the last post I wrote until now) end of April 2013, the team behind the software didn’t sit still.

Next to C# the tool now can read and understand VB.NET (looks logic, it make use from the same Rosylin project to compile like C#) and JAVA.

I’m hardly write any code and those languages so I lost track of the latest developments of the codicesoftware team until I received a tweet to attract my attention to their latest release.

What’s a visual merge representation?

Was the first question that raised my mind after reading the title of the blogpost “SemanticMerge goes Visual”. The first picture already revealed a bit of the answer.

imageNext to the textual representation, that already existed in the previous versions, where you get a list of detected changes in the source and destination file.

In the screenshot on your right you’ll see they make use of symbols to decorate a change ( C ), a move ( M ), a renaming ( R ) or deletion ( D ) in source or destination file.

It’s gives you a readable and logic overview before you start merging the files together.

The Visual representation will extend the textual view. Instead of a list of changes you’ll get the 3 files that are compared during a merge (source, base and destination).

image 

Watch especially the same symbols that were used in the text overview. On the base class you’ll see that they placed the symbols at the side where the change was.

image

Together with the arrows drawn to guide the change you’ll get a pretty good overview of the merge changes that were discovered by the software.

Remarks

Although I like this representation to help you understand what is being merged there are some areas where some improvement can help the end user.

Unchanged items

The examples that were installed with the software have classes and methods that all were changed, moved or deleted. It’s a perfect way to show the merge capabilities of the SemanticMerge tool but for the visual representation it hides the unchanged code that in a real life example will be there (we hardly refactor the whole file at once).

image

The screenshot on your right shows a test class we have made to test some of the functionality without having to implement WCF services that would be used to access the final code.I just took a diff in this example (using the SemanticMerge diff viewer that also supports the visual view)

Only a few methods are changed, they get lost in the many unchanged methods or code.

This is the default view when opening the visual representation. You have a few buttons on top to hide the unchanged items or to group them so you’ll have a better view. Why not making that the default presentation. How many developers are interested in code that didn’t changed when they are merging branches together. My attention always go to the changes and verifying that those are merged correctly in the destination file.

image

image

I think it would be better to have the end user choose their default view when merging.

image

Leave the buttons on the top so we can switch when necessary but don’t let us click to much to reach our goal.

Picture size

The first time I opened the Visual view it was big, too big. it really slammed me in the face, and on top of that, the picture was to big for the opened window so I had to use the scrollbars to get a complete view.

The view support zooming by the ctrl key and the mouse wheel but I couldn’t find a zoom button or slider (handy when working on a laptop without external mouse). Nor did the combination ctrl key and the plus or minus key worked to zoom.

Make the diagram fit the window (even a bit smaller) and together with the hidden unchanged items would make it a lot better.

Go to change

When you select a change in the textual view you’ll have the choice to view the code on source or destination or the differences between the two depending on the change.

image

I can’t find the same functionality in the visual view. You can select the change but clicking or double clicking did nothing. I thought it would bring me to the correct place inside the code file of the source, if clicked on the source or the code file of the destination if I clicked the destination representation.

Now I can see that a certain method is changed but I have to go back to the textual overview before I can view the source (or destination) code block.

Hidden functionality

I find the visual representation a bit hidden in the SemanticMerge tool. Ok, the button states “Visual Merge” but wouldn’t a small preview of the visual presentation attract more users to use the presentation? Be proud on what you accomplished, a bit showing off wouldn’t hurt anyone.

image

Conclusion

I definitely find a visual representation of the merge path a big plus in a merging tool. Certainly when there are a bunch of changes on the file (changes, moves, deletions, …) the visual view will help you understand what will be merged.

Just as last time I’m impressed by the fine work of the team and how they change the way of merging code away from the painful process it used to be.

Some small changes (see above) would make the software a lot better and ready for use.

Take a look for yourself, download SemanticMerge, use it today, you’ll find it very handy especially in larger teams!

Semanticmerge diff viewer settings for SVN

Last week I blogged about the new merge tool that understands your code and helps you fight the merge agony when working with multiple developers on a project.

Diff viewer

The semanticmerge tool doesn’t only helps you with merges, you can use their diff viewer to compare the changes you made on a file, or compare the changes you made against any version in you source control.

Setting Tortoise SVN options

To use their diff viewer we’ll have to tell Tortoise SVN witch files he have to compare. Right click on a folder and choose for ‘TortoiseSVN’ and then settings.

In the settings screen go to ‘External Programs’ – ‘Diff Viewer’. There you can set the path to the semanticmergetool.exe.

image

For the diff viewer we’ll only need to set 2 options. The ‘base’ file and the ‘mine’ file (where the changes are. The complete command:

Testing

After saving these settings in TortoiseSVN we can start testing. Open up one of your code folders and right click on a file you’ve changed.

image

Click on ‘TortoiseSVN’ – ‘Diff’ and wait a second for semanticmerge to come up. One of the advantages of the semantic diff viewer is that you immediately get an overview of the semantic differences on the left hand side.

image

You’ll see directly if parts are moved or changed. You can click on one of the changes and click on the diff button to only see that change.

image

image

Or you can click the launch diff tool so you can scroll through your file and see all changes. You still have the option to open the specific change in the overview.

Conclusion

I was already impressed with the merge tool an sigh but the also the diff tool has some advantages you don’t find in the basic Tortoise diff viewer.

DISCLAIMER: The configuration seems to be working on my system. I’m not 100% sure it’s the correct configuration. Use it on your own risk!

New merge tool Semanticmerge, tested on SVN

This morning I saw a post of Scott Hanselman passing by on facebook:

“There’s as new code merging tool in town… “Semantic Merge – Plastic SCM” http://t.co/mlaqWnVGST Free Download of the Beta”

The welcome text on their site is very promising:

“a semantic merge tool…
that understands your code”

A project we’re working on at work is in a phase of a lot of refactoring’s. the base functionality seems to be working fine, we’re now taking care of better fault handling, better logging, better messaging, …

That includes a lot of refactoring’s, moving methods, adding log messages, updating methods in a team of 3 developers. That results in a lot of merge errors as we more then once make changes to the same files in the project.

The default text merge tool can’t handle these changes and more then once we’ve been fouling around for over an hour to get the files back to a ‘working’ state.

No SVN documentation

We still work with SVN for our internal projects. For my personal projects I’m hooked up to git in combination with GitHub as you probably can see in the different topics I blogged about before.

The guys from Semanticmerge placed some documentation how to configure the merge tool for git, TFS and Plastic SCM but they didn’t include documentation about configuring TortoiseSVN (the tool we are using).

On their UserVoice page there’s already a request for documentation on SVN and Tortoise.

Trial en error configuration

I didn’t wanted to wait for the upcoming documentation and decided to try and figure it out myself. I took the documentation for TFS for the semantic mergetool, the Diff/Merge configuration documentation on  MSDN, the Tortoise documentation (search for merge tool) and a post on stackoverflow to combine the different parameters that have to be set.

After some attempts I think I got it running.

Setting Tortoise SVN options

First start with downloading the beta of the Semanticmerge tool and install it on your system. You’ll need the location where you installed the tool, note it during installation or you’re going to have to search your system for the executable needed in the next step.

In TortoiseSVN you can set the merge tool you want to use (default TortoiseMerge), this we’ll have to change to the semanticmerge tool. Right click on a folder and choose for ‘TortoiseSVN’ and then  ‘Settings’.

image

In the settings screen go to ‘External Programs’ – ‘Merge Tool’. There you can set the path to the semanticmergetool.exe.

image

If you would try to merge now you’ll get an error message. The tool will be started but it doesn’t know witch files he have to merge. Those parameters will we have to add after the path to the executable.

After some searching I added following options:

So the complete command you have to fill out for the external merge tool:

Testing

I’ve created a test class with 3 simple methods and committed this to our SVN server.

On one virtual machine I updated the source and added some changes in the file. I moved the second method under the third method and added some dummy code.

I then committed the file back to the server. On my machine I also changed the same file. I’ve added a test class under the existing class and added some dummy code to the second method (that is still the second method in row).

When I then update my code on my machine I’ll see we have some merge errors (like expected).

image

If you then click on ‘Edit conflict’ the merge tool appears.

image

It indicates that Method2 is changed on both contributors. You can click the merge button to view how the tool automatically merge the files.

image

The moved method is nicely merged with the changes of the not moved method. Just like we wanted. The merged result:

Conclusion

At first sight (the merged class wasn’t that difficult) the tool does what it has to do. I’m going to start using it on our project now and will see if some more complex merges will have the same result.

One of the problems I’m still facing is that for SVN the file doesn’t seems to be resolved automatically. I still have to find the file and right click and choose ‘resolved’. Or you can do that on a checkin but then you have to remember witch files you’ll already merged.

It can be it’s another parameter you’ll have to add to the command line expression but I couldn’t find any documentation.

DISCLAIMER: The configuration seems to be working on my system. I’m not 100% sure it’s the correct configuration. Use it on your own risk!

UPDATE: You can also use the samentic merge tool as diff viewer in TortoiseSVN.

Securing Backend Pages in MVC for Umbraco 4.11.1

Beginning of the year I posted an article on how you can create backend pages in the open source ASP.NET CMS Umbraco (http://blog.bartdemeyer.be/2013/01/using-mvc-backend-pages-in-umbraco-4-11-1). At the end of the post I published a little TODO list:

  • Block access to the backend controller for non authenticated users
  • Adapt the actions that you get in the Umbraco back end by right clicking
  • Due to a very busy work schedule I didn’t had the time to look into this list. In the mean while version 6 was released by the Umbraco core team.

    Last week I received the question if I had already found a solution for the first item on the todo list: blocking access to the backend controller for non authenticated users.

    Between the search for Easter eggs together with my son, backing pancakes and nothing interesting on the TV this evening I found the time to implement the feature.

    Reminder: the problem

    A small reminder of the problem we are facing. We’ve created a MVC controller to access from the Umbraco backend to perform our own business logic like shown in the image below.

    image

    We used the implementation if the Umbraco’s Surface  Controller that will create a MVC area. If we know the structure of the URL to this area – controller – action we can access the page without logging in the Umbraco backend.

    image

    This is off course a security risk we have to solve. We don’t want non authenticated users to be able to access our business logic.

    Using the Authorization attribute

    In ASP.NET MVC access to certain controllers or individual actions is managed by using the Authorization annotation attribute. You can use the default attribute where you are able to filter request based on specific users or specific user roles.

    But you can off course inherit from the default attribute and implement your own authorization attribute.

    Create the UmbracoAuthorizeAttribute

    In this post I’ll extend the project we’ve created during the previous post and that you can find on Github.

    Add a folder ‘Helpers’ in the UmbracoMVCDemo.BackEnd project. In that folder create a new class ‘UmbracoAuthorizeAttribute’. Add the public keyword to the class implementation and we shall inherit from the System.Web.MVC.AuthorizeAttribute.

    Implementing authorization

    Implementing your own authorization login can be done by overriding two methods from the base class: AuthorizeCore(HttpContextBase httpContext) and OnAuthorization(AuthorizationContext filterContext).

    In AuthorizeCore(HttpContextBase httpContext) we will connect to the Umbraco backend to check if the user is authenticated or not.

    I’ll first check if the httpContext contains any value, if not we’ll throw an exception. After that I’ll use a method from the Umbraco BasePage: ValidateUserContextID. This method will return true if the user is authenticated and false if not. With that result we’ll set a class variable we can use in the implementation of the OnAuthorization method.

    In this implementation we’ll first execute the base class implementation before we check the failedAuthentication class variable. If the authentication in the failedAuthentication failed we’ll send the user to the login page using the FormsAuthentication class.

    Adding authorization to our controller

    It’s not enough to create the attribute, we still have to tell our controller to use the authentications we’ve implemented. Open up the DemoAdminSurfaceController class we created in the previous post and add the UmbracoAuthorize attribute (line 4).

    Testing

    I’ve opened up another browser and accessed the same action again. (keep in mind that we make use of the default FormsAuthentication that will use a session cookie. Therefor will a new tab in the same browser not give the desired effect). Because we’re not authenticated we get redirected to the login page.

    image

    You’ll see the url we entered in the ReturnUrl parameter. If we now enter our credentials we’ll get redirected to the Umbraco back end. Although the ReturnUrl is filled out we’ll not be redirected to the correct page. We still have to click on our demo Section and demo node.

    Remark

    I’ve had to alter the default web.config file to be correctly redirected to the login page. I’m not sure this needs to be done if you have a fresh installation, I’ll add it here to avoid bad redirections.

    Source

    You can find the full source code from this post and the earlier posts on Github. Feel free to download, fork, comment, …

    Roadmap for ASP.NET MVC, ASP.NET Web API, and ASP.NET Web Pages

    I saw a tweet from John Galloway (@jongalloway) coming by this evening that can interest any developer in the ASP.NET stack. It’s a draft of the features that Microsoft want to include in the next release of the stack.

    There are no release dates and they do emphasize it’s a planning document but I like the way Microsoft is going with the open sourced ASP.NET stack.

    http://aspnetwebstack.codeplex.com/wikipage?title=Roadmap

    Creating a JIRA interface in 1-2-3

    For one of our projects where close to the first test release. While preparing the necessary documents (test overview, legal documents, bug report, …) I felt a bit ashamed to give the client a Word document to report bugs to us. For a software development company it’s a bit shameful to let the client fill out a paper, scan it, mail it and then have someone copy the document in our issue tracker.

    While driving home I thought it couldn’t be that difficult to set up a small site where the client can report the bugs. But I didn’t want to create a new bug tracker nor give the client direct access to our issue tracker (JIRA from Confluence).

    JIRA has a API that you can address to create and fetch issues from their tracker. But I didn’t want to  spend a couple of hours implementing the REST service. (jeah, I know, lazy as hell)

    Nuget to the rescue

    When I got home, the first thing I checked was the nuget site to see if anyone had already created a package to communicate with the JIRA API. First thing that showed up in the search was Atlassian.SDK a complete package, even including a Linq Query implementation to search for issues created by Federico Silva Armas.

    image

    Great, let’s get started.

    New MVC 4 web application

    I fired up Visual Studio 2012 and choose a new MVC 4 Default application with Razor support. Opened up the Package Manager Console and ran the following command:

    Add controller

    I added a new controller in the controllers folder and named it HomeController. In the Index method I’ve added the call to JIRA to fetch all items from a certain filter.

    The implementation is really simple, create a new Jira object with as parameters the URL to your JIRA instance, a username and a password. (in the constructor)
    In the Index method you then can call GetIssuesFromFilter and add the filtername as parameter. The advantage of using a filter is that you can always change this filter settings if you want to remove certain issues from the result without to change anything in your application.

    You’ll need off course the user rights correctly set in JIRA to access the issues that are returned from the filter.

    Add View

    Next I created a new view under the Views – Home folder using the add view action after a right click on the folder. Create a strongly-typed view by selecting the Atlassian.Jira class as Model class.

    image

    Click Add and the view is created.

    If you hit F5 now, your browser should open up and you get already the overview screen for all issues returned from the filter you have selected.

    In 10 minutes I’ve created a working (but ugly) web application that can fetch all the tickets I wanted to show to the customer. Of course your view will need some tweaking to show the values of the issues you want to be shown.

    Adding a new issue

    Adding a new issue to JIRA is as simply. (You can check the complete code on Github). I give you a (very) short overview.

    Create 2 new methods Create in the HomeController. The first one just returned the view, the second one add the HttpPost attribute and this will receive an Atlassion.Issue object with all the values. Pushing this to JIRA is as simple to create a new issue object from the jira object and fill up the values. Hit SaveChanges on the jira object and your issue is created.

    On thing I had a problem with is the projetc name – key difference in the jira.CreateIssue() method. I first tried with the project name but I got some HTML 500 error returning. Although for fetching issues this worked fine but for creating issues you apparently have to gibe the project key instead of the project name.

    Look and feel

    Ok, we have a working application but the look and feel isn’t that, certainly not enough to send to an end user. I didn’t wanted to spend to much time on the look and feel and decided to use the Twitter Bootstrap.

    Don’t start downloading the package from the Twitter Bootstrap website. Instead use the nuget package manager again. There is a Bootstrap for MVC 4 package that will add all necessary items to your solution.

    Open up the Package Manager Console again and run the following command:

    Add bootstrap to the bundles

    To add the bootstrap files to your application we’ve to alter the style and script bundles that are in use. Open up the BundleConfig.cs class in the App_Start folder. There we can add the necessary files to the bundles.

    Online 28 I changed the the Content/css bundle to include the bootstrap css instead of the default Site.Css. On line 21 I added a script bundle to include the bootstrap.js file.

    Because I added a script bundle we’ll have to add this bundle in the _ Layout.cshtml view.

    Just underneath the rendering of the jQuery bundle add the rendering of the bootstrap bundle. (at the bottom of the view.

    Result

    I was able to create an new interface for our client to report bugs in just an hour of two. (the above only took 30 min but I’ve added some extra logic to send emails and stuff). Off course it’s not correct to add all business logic in your controllers and to hard code some strings and stuff. For this small application it’s more then enough. Underneath the screenshots of the 2 pages (in Dutch).

    image

    image

    Source code

    You can find this project on Github, feel free to fork, comment, …

    One note, to avoid adding my personal credentials to the project I’ve created an xml file that holds the credentials in the App_Data folder. When creating the Jira object I fetch the values from this file. For obvious reasons I added this file to the .gitignore so it wouldn’t be pushed to Github. The structure of the file:

    Creating graphs in WPF using OxyPlot

    For one of our projects we had to retrieve data from an external source and store them in a database. One of the requests from the client was to see the retrieved data in a graph. Furthermore, he wanted to see a live update every second of the retrieved data in that graph.

    After some searching I found the OxyPlot library. This library can be used in WPF, Silverlight, Windows Forms and even in Windows Store apps. The are now working on a alpha release for  Mono.

    Although the package is already downloaded more then 10 000 times there are not so many blog posts to find about implementing the library.

    Using Nuget

    The easiest way to include OxyPlot in you application is by using the Nuget package manager in Visual Studio.

    Open op Visual Studio and start with creating a new WPF project. Choose a name and a location and hit ‘OK’.

    image

    After the project is created open up the Package Manager Console and type following commands at the prompt and hit enter (after every command):

    • Install-Package Oxyplot.Core
    • Install-Package Oxyplot.Wpf

    image

    You can of course use the Package Manager UI by right clicking References and choose Manage Nuget Packages and them that way.

    Create the ViewModel

    We’ll use the MVVM model (partial) to render the Graph on the screen. First step is to create the ViewModel for our MainWindow.xaml. Right click on the project and add a folder ViewModels. Right click the folder and add a class MainWindowModel.cs.

    image

    To use the MVVM model we need to make the class public and inherit the INotifyPropertyChanged interface.

    NOTE: The OxyPlot WPF package doesn’t completely support the MVVM model. You can’t use only the OnPropertyChanged method to update the graph. You’ll still need the manually refresh the graph from the code behind class of the XAML page. I saw a post in the discussions on codeplex that they are looking to add complete MVVM model but are short of time at the moment. Maybe the next version will support it. To be complete I add the necessary steps to implement the MVVM model in this post.

    After inheriting the interface we have to implement the PropertyChangedEventHandler event as shown below.

    Now we can add some public properties that we’ll bind to in the XAML page. First create a PlotModel that’s part of the OxyPlot library. In the constructor we’ll initiate the PlotModel parameter. The setter will call the OnPropertyChanged method that will notify the view there is a change on the object that may have to be rendered.

    Adding the graph to the page

    We can now add the graph to the MainWindow.xaml page. Before we can add the graph we’ll need to add the namespace for the OxyPlot library (line 4). We’ll bind the PlotModel parameter to the PlotModel we’ve created in the view model.

    Binding the model to the view

    Last part for our setup is to bind the view model to the view. Because we don’t use any MVVM frameworks we’ll have to manually bind the model in the code behind of the MainWindow.xaml page.

    Add a private property of our view model and initiate this property in the constructor. Point the DataContext to this property and we’re done.

    If we hit F5 to run the application we’ll see an empty window opening. We have of course add some data to the PlotModel before the graph will be rendered.

    Set up the Graph – PlotModel

    Back to our view model to set up our view model. First start will be adding the axes for the graph so OxyPlot knows where to plot the points.

    Create a new method SetUpModel in our view model class. In here we’ll define the legend of the graph (position, border, sire, …) and add 2 axis. A DateTimeAxis for our X-axis (we want to plot points in a line of time) and a ValuaAxis for our Y-axis.

    The set up for the legend of the graph is self explanatory. At line 10 you’ll see the creation of the DateTimeAxis. We’ll choose the position of the axis, give it a name (Date) and a string format pattern how the date has to be displayed. Next to that we’ll add some parameters to show major and minor gridlines and the interval size. After creation we’ll add the axis to the PlotModel.
    The ValueAxis is similar initiated, only did we position the axis on the left side and tell the Axis to start at the value 0.

    If we add a call to this method in the constructor of our view model and hit F5 again we’ll now see window opening that contains an empty graph.

    image

    The data

    Now we have our graph and are ready to add some graph lines to the PlotModel. In this post we’ll add four different lines that will be plot.

    To avoid complexity I added a class Data.cs where the data is hardcoded.  In a real life application you’ll be implementing a database call or a service invocation or a web API request, …

    The data I’ll receive will be a List<T> of a new simple class I’ve created: Measurement. This class has 3 public parameters: a DetectorId (long), a Value (int) and a DateTime (DateTime). I’ve added 4 different detectors that each have 10 measurements of a random value between 1 and 30.

    Add the data to the PlotModel

    In the view model we’ll create a new method LoadData where we’ll get the data and add some code to plot all point on the graph.

    After fetching the data I’ll use the LINQ GroupBy expression to create a IEnumerable with as key the DetectorId and a collection of Value and Datetime as values. (line 5)

    This will allow us to loop over the result and then add a LineSerie per Detector. In the setup of the LineSerie we’ll set some properties like colors and line thickness and the title. (line 9 till 18).

    After we’ve created the LineSerie we can add the points to the LineSerie. We make use of the DateTimeAxis builtin funtion to convert a date to a double and add the value (line 20).

    After we added the points to the LineSerie we’ll still have to add the LineSerie to the PlotModel. (line 21).

    And that’s it. Hit F5 and you’ll see that are graph will be rendered and that per detector a line will appear plotting the random values between 1 and 30.

    image

    Updating the graph real-time

    For the second question of our client we had to add real-time updates. For this demo I’ll add an new measurement to the graph every second. The OxyPlot library will make sure our graph will move along every time we’ll add a new measurement.

    I’ve added a static method in the Data.cs class that will return a random value a second after the DateTime parameter that will be send along. In the view model I’ve created a private parameter DateTime parameter (lasUpdate) that will hold the moment we last updated the graph.

    Update the view model

    We’ll add a new method to the view model UpdateModel. This time we’ll make it public so it can be called from the view as shown in the next chapter.

    In the UpdateModel method we’ll fetch the data from out Data.cs class and perform the same grouping action as in the LoadData method.

    This will allow us to fetch the data per detector. After retrieving the correct LineSerie we can add the points like we did in the LoadData method.

    Update from the view

    We don’t want the update of the model to initiate before the previous is completely rendered to avoid an exception that we are modifying a list while rendering the graph. Therefor we make use of a  CompositionTarget.Rendering event. This event will be fired every time the view is done rendering.

    In the constructor of the MainWindow.xaml.cs class we’ll attach an eventhandler on the Rendering event.

    In the event handler we’ll call the update method from the view model. After that call we’ll have to tell the PlotModel that there was an update and that he has to refresh the graph (line 14). (This is where OxyPlot drifts away from the MVVM model).

    Avoid to many updates

    The rendering event is ideal to avoid exceptions but you’ll see to many updates per second to keep the graph readable. We can solve this to add a StopWatch in the code behind and only trigger the update when the last update is at least a second ago.

    After adding the StopWatch we’ll see that every (for this demo 5 seconds) a new point is added to the graph.

    image

    Source code

    As you can see, with OxyPlot you don’t need to write to many code to create real time updating charts.

    OxyPlot has many more options and graph styles, they have provided a demo page where a variety of graphs are rendered in Silverlight. You can copy the source code with the keyboard combination Ctrl+Alt+C. If you want to copy the properties use Ctrl+Alt+R.

    The source code of this post can you find on Github. Feel free to fork, download, …

    The hard search for missing Extension methods in Umbraco 4.7.0

    Implementing SignalR

    For one of our company websites we got the question from HR to implement a chatbox on the website so interested candidates could contact them in an easy way. After comparing a few open source alternatives we decided to build our own using the just released SignalR framework.

    Everything worked fine on our development server where we’re running a backup of the production Umbraco website. Our dev team could easily create the chatbox and sooner as expected we had a working chatbox to present to HR.

    Moving changes to staging

    Like all our development projects we work in different steps. Development on the local machine or development server, client tests on the staging machine. This way we remove the risk that some developers test come in the way of our clients testing.

    For this implementation the client is our own HR department, still we follow the same development guidelines to avoid test data on the acceptance (staging) environment.

    Extension methods missing

    After copying all DLL’s and usercontrols to the staging environment we noticed that all of our Macros in Razor where failing. Suddenly the ‘umbraco.MacroEngines.DynamicNodeList‘ was missing the extension methods like ‘First’ and ‘FirstOrDefault’.

    image

    Start debugging

    First reaction was of course to delete all the DLL’s we’ve altered for this last release and to reinstate the old DLL’s. Still we faced the same problem.

    Back to start, we took a new backup from the production server and started the deploy again. There we noticed we overwritten a third party DLL ‘Newtonsoft.Json.dll’, a well known library for Json conversions created by James Newton-King. More information on his website.

    We skipped the replacement of the DLL and all of our Razor scripts we’re working like before. Problem solved (we thought).

    SignalR failed to work

    After the re-deploy of our chatbox DLL’s and usercontrols we couldn’t get the chatbox to work. It seemed the SignalR hub was missing to contact the server to register a chat user. A check with Firebug reveled the hub couldn’t be loaded because a DLL reference couldn’t find the correct version of the DLL. Of course the ‘Newtonsoft.Json.dll’ was playing up again.

    image

    Some more debugging

    No problem, we find out where Umbraco references the DLL and update it to the latest version and rebuild the Umbraco DLL and we’re good to go. Wrong, we checked the source of the 4.7.0 version that was available on codeplex but no sign of ‘Newtonsoft.Json.dll’.

    Then it had to be one of our own projects referencing the DLL? No, none of our own project had a reference to the ‘Newtonsoft.Json.dll’.

    I loaded the project on disk, configured it in IIS and attached the Visual Studio debugger. In the Debug menu –> windows –> Modules component I could find the ‘Newtonsoft.Json.dll’ reference but no way to find out witch component was responsible for loading the DLL.

    Some more (manual) debugging

    As I couldn’t find a way to find out who was responsible for loading the DLL I had only one option left.

    I took a look at the download at codeplex and removed all DLL’s that were not in the original download. Of course some other problems showed up as all custom created usercontrols failed to load but the Razor Extension methods worked and I could start a tedious process of adding DLL’s to the bin folder with my Visual Studio debugger attached and to reload the site every time.

    Found the problem

    When I copied the ‘Twitterizer2.dll’ back into the bin folder the Razor scripts failed again. I finally found the source of the problem. After a quick look at the Github repository of the Twitterizer project I found the Newtonsoft references and the different versions in the packages folder.

    Resolving the problem

    I did found the problem but we wanted to keep the Twitter possibilities that we had implemented before so we needed the Twitterizer2 DLL.I forked the project on Github and cloned the project locally.

    After updating the package to the latest version (the same our SignalR implementation was using) I’ve rebuilded the DLL and copied it to the Umbraco bin folder. Finally we had our Razor scripts working and our chatbox was responding!

     

    It took me half a day of debugging before I did found the problem. If anyone has a good way to find out witch component loads a certain DLL, let me know in the comments or by a tweet, …

    Using MVC backend pages in Umbraco 4.11.1

    Umbraco backend

    In a previous post we saw the use of your own MVC controllers in the Umbraco front end. The Umbraco community has made effort to document these possibilities on our.umbraco.org. The use of MVC in the back end of the Umbraco interface is not very well documented and it took me some analysis and debugging of the source code to find out how this could be implemented.

    Default you have 7 sections in the backend:

    • Content
    • Media
    • Settings
    • Developer
    • Users
    • Members
    • Translation (hidden by default for the admin user)

    These sections will do in a plain website where you don’t want to use custom code.

    One of my personal projects is to create a new website for the sport club where I’m already 15 years affiliated with. The current website is already 7 or 8 years old and was written in PHP. Needless to say that the look and feel is outdated, the PHP code I’ve written 8 years ago wasn’t the most beautiful piece of code I’ve created.
    I wanted to create a new website in ASP.NET MVC but I don’t want to put the effort in creating controllers and views for the administration part where a good CMS can fill up this gap.

    Because I’m already familiar with Umbraco it looked a good choice to combine the strength of Umbraco with custom development. But therefor I had to get custom MVC controllers working in the backend.

    New backend project

    This blog post goes further on the project created in the previous post. You can download the source at Github.

    In the existing solution we’ll add a new project by right clicking the solution and choose Add – New project.

    image

    Choose for a ASP.NET MVC 3 Web Application and give it a name (in this demo: UmbracoMVCDemo.BackEnd)

    image

    In the next window choose for Internet Application and the Razor view engine. We’ll need the core Umbraco libraries so open up the Package Manager Console and type ‘”Install-Package UmbracoCms.Core” and hit enter. Wait for the “Successfully added ‘UmbracoCms.Core 4.11.1’ to UmbracoMVCDemo.BackEnd.” message and we’re ready to go.

    Adding a new section

    First we’ll have to create a new section in the Umbraco backend. Open op your windows explorer and go to the root of the Umbraco installation. Open up the config folder and open up the applications.config file.

    <?xml version="1.0" encoding="utf-8"?>
    <applications>
      <add alias="content" name="Content" icon=".traycontent" sortOrder="0" />
      <add alias="media" name="Media" icon=".traymedia" sortOrder="1" />
      <add alias="settings" name="Settings" icon=".traysettings" sortOrder="2" />
      <add alias="developer" name="Developer" icon=".traydeveloper" sortOrder="3" />
      <add alias="users" name="Users" icon=".trayusers" sortOrder="4" />
      <add alias="member" name="Members" icon=".traymember" sortOrder="5" />
      <add alias="translation" name="Translation" icon=".traytranslation" sortOrder="6" />
    </applications>
    
    

    You’ll see the existing 7 sections configured in the config file. We’ll add a new one:

      <add alias="demo" name="Demo" icon="demo.gif" sortOrder="7" />
    

    We give the new application or section the alias ‘demo’ and the name ‘Demo’. For the icon we’ll add a new picture called ‘demo.gif‘ and set the sort order to 7. (The default images are showed using sprites where the icon name is actually the css class to define the background image. You can choose to alter the existing sprite and add a css class or select new picture). Create a new picture with your favorite image editor, name it ‘demo.gif’ and place it in the folder: [Umbraco root]/umbraco/images/tray/ folder.

    We’ll have to force the application pool to restart before Umbraco loads the new section. Open up the web.config file in the umbraco installation root and add a line break, save and refresh the page.

    Still the new section is not available in the backend because none of the users have access to this new section. Go to the users section, open up your user and add a check in the sections area next to the demo section. Refresh your page and you’ll see the new section appear.

    image

    When you click the Demo section at the bottom of the page we’ll see a new section opening up with an empty tree and the section name in square brackets.

    Adding a tree

    Next step will be to create the tree for our Demo section. This tree you can populate with data you choose. For this demo we’ll use some dummy data.

    In our backed project add a folder ‘Trees’ and add a new class named DemoTree. This class will have to inhered from the umbraco.cms.presentation.Trees.BaseTree class. This class has 3 abstract methods that we’ll need to implement.

    • RenderJS: the javascript that will run when a tree node is clicked
    • Render: the creation of tree nodes
    • CreateRootNode: to define the root node.

    We’ll also have to create a constructor that will receive a string that will forward to the base class.

    Last but not least we’ll have to decorate the class with the umbraco.businesslogic.TreeArrtibute where we define the application alias and title.

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using System.Web;
    using umbraco.cms.presentation.Trees;
    
    namespace UmbracoMVCDemo.BackEnd.Trees
    {
    	[Tree("demo","demo","Demo")]
        public class DemoTree : umbraco.cms.presentation.Trees.BaseTree
        {
            public DemoTree(string application) : base(application)
            {
            }
    
            public override void RenderJS(ref StringBuilder Javascript)
            {
                throw new NotImplementedException();
            }
    
            public override void Render(ref XmlTree tree)
            {
                throw new NotImplementedException();
            }
    
            protected override void CreateRootNode(ref XmlTreeNode rootNode)
            {
                throw new NotImplementedException();
            }
        }
    }
    

    Let’s start with defining the root node what is as simple as setting the NodeType and NodeId parameter.

    protected override void CreateRootNode(ref XmlTreeNode rootNode)
    {
    	rootNode.NodeType = "init" + TreeAlias;
    	rootNode.NodeID = "init";
    }
    

    Next we’ll implement the Render method and attach 2 nodes to the tree.

    public override void Render(ref XmlTree tree)
    {
    	XmlTreeNode xNode = XmlTreeNode.Create(this);
    	xNode.NodeID = "1";
    	xNode.Text = "Demo Node 1";
    	xNode.Action = "";
    	xNode.Icon = "folder.gif";
    	xNode.OpenIcon = "folder_o.gif";
    	OnBeforeNodeRender(ref tree, ref xNode, EventArgs.Empty);
    	if (xNode != null)
    	{
    		tree.Add(xNode);
    		OnAfterNodeRender(ref tree, ref xNode, EventArgs.Empty);
    	}
    
    	XmlTreeNode xNode2 = XmlTreeNode.Create(this);
    	xNode2.NodeID = "2";
    	xNode2.Text = "Demo node 2";
    	xNode2.Action = "";
    	xNode2.Icon = "folder.gif";
    	xNode2.OpenIcon = "folder_o.gif";
    	OnBeforeNodeRender(ref tree, ref xNode2, EventArgs.Empty);
    	if (xNode2 != null)
    	{
    		tree.Add(xNode2);
    		OnAfterNodeRender(ref tree, ref xNode2, EventArgs.Empty);
    	}
    }
    

    In the RenderJS method remove the ‘throw new NotImplementedException” line and build the project. We’ll have to copy the UmbracoMVCDemo.BackEnd.dll from the bin folder to the Umbraco installation bin folder.

    Refresh the page and you’ll see two new node appear. Notice that the Demo root node isn’t surrounded by square brackets any more.

    image

    If you open up the trees.config file in the umbraco root – config folder you’ll see Umbraco automatically added the tree.

    For now the nodes don’t do much if you click on it because we left the Action attribute empty. Before we can add an action we need the editor for the right hand side.

    Adding an editor – controller and views

    Like I said before we want to create the editor in a MVC pattern instead of the default Webforms like was possible before. Therefor will create a new controller by right clicking the controllers folder in the backend project.

    image

    Name the controller DemoAdminController and choose for the empty controller template. Because we’ll already have a DemoController in the frontend application I named this one DemoAdmin to avoid confusion.

    The Controller will have to inhered from the Umbraco.Web.Mvc.SurfaceController before we can use it in the backend.

    IMPORTANT: Although the documentation of version 5 stated the controller didn’t need to be suffixed by ‘Surface’ I noticed there are constraints set on the PluginControllerResolver in the Umbraco source code. We can remove this constraint in the source code but the easiest way is to rename our controller to DemoAdminSurfaceController. This will change our URL’s but they won’t be visible in the Umbraco backend anyway.

    We have to decorate the class with the Umbraco.Web.Mvc.PluginController attribute. (yes, the plugincontroller attribute although we’re inheriting a SurfaceController). In this attribute we’ll have to enter the area name. Every plugin will be placed in his own MVC area to avoid duplicate views and controllers. We’ll set the area name to ‘demo’.

    Last but not least we’ll have to create two constructors. One without parameters, one that will take a Umbraco.Web.UmbracoContext instance and forward this to the base class.

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Web;
    using System.Web.Mvc;
    using Umbraco.Web;
    using Umbraco.Web.Mvc;
    
    namespace UmbracoMVCDemo.BackEnd.Controllers
    {
        [PluginController("Demo")]
        public class DemoAdminSurfaceController : SurfaceController
        {
    
            public DemoAdminSurfaceController()
            {
            }
    
            public DemoAdminSurfaceController(UmbracoContext umbracoContext)
                : base(umbracoContext)
            {
            }
    
            public ActionResult Index()
            {
                return View();
            }
        }
    }
    
    

    Last but not least we’ll have to create a view to back up the Index action on our controller that will for now will be one line:

    “<h1>Backend Demo Controller – view</h1>”

    Build the backend project and copy the dll to the Umbraco installation bin folder. Default will the view be searched in the following folder:

    [Umbraco ROOT]/App_Plugins/Demo/Views/DemoAdminSurface/. That is App_Plugins/[AreaName]/Views/[ControllerName]/. Create the folders and add the index view.

    I we go to the Umbraco backend we”’ll notice the 2 demo nodes still don’t do anything if we click them. We’ll first have to assign an action to these nodes.

    Assign an action to the nodes

    To assign an action to the nodes we’ll have to do two things. We’ll have to implement the RenderJS method in out Tree class and fill out the Action parameter from our nodes in the Render method.

    In the RenderJS method will render the necessary javascript that will open our editor in the right hand pane by using the ‘UmbClientMgr’ javascript object.

    public override void RenderJS(ref StringBuilder Javascript)
    {
    	Javascript.Append(
    	@"function openDemoController() {
    		UmbClientMgr.contentFrame('/Demo/DemoAdminSurface');
    	}");
    }
    

    In the Render method will add a call to the javascript action we just created (for now only on the first node).

    public override void Render(ref XmlTree tree)
    {
    	XmlTreeNode xNode = XmlTreeNode.Create(this);
    	xNode.NodeID = "1";
    	xNode.Text = "Demo Node 1";
    	xNode.Action = "javascript:openDemoController();";
    	xNode.Icon = "folder.gif";
    	xNode.OpenIcon = "folder_o.gif";
    	OnBeforeNodeRender(ref tree, ref xNode, EventArgs.Empty);
    	if (xNode != null)
    	{
    		tree.Add(xNode);
    		OnAfterNodeRender(ref tree, ref xNode, EventArgs.Empty);
    	}
    
    	XmlTreeNode xNode2 = XmlTreeNode.Create(this);
    	xNode2.NodeID = "2";
    	xNode2.Text = "Demo node 2";
    	xNode2.Action = "";
    	xNode2.Icon = "folder.gif";
    	xNode2.OpenIcon = "folder_o.gif";
    	OnBeforeNodeRender(ref tree, ref xNode2, EventArgs.Empty);
    	if (xNode2 != null)
    	{
    		tree.Add(xNode2);
    		OnAfterNodeRender(ref tree, ref xNode2, EventArgs.Empty);
    	}
    }
    

    Build the project and copy the dll to the Umbraco installation bin folder and refresh the backend in the browser. If we now click the first test node we’ll see the test view appear in the right hand side frame.

    Automating the copy process

    Just like in the previous blog post you can use the build events to automate the copy process of DLL’s and views. Place the 2 xcopy commands in the post build event.

    xcopy $(TargetPath) $(SolutionDir)UmbracoMVCDemo\bin\ /C /Y
    xcopy $(ProjectDir)Views $(SolutionDir)UmbracoMVCDemo\App_Plugins\Demo\Views\ /E /Y

    Source code

    You can find the solution above on Github.

    To do’s

    There are still some things I need to resolve:

    • Block access to the backend controller for non authenticated users
    • Adapt the actions that you get in the Umbraco back end by right clicking
    • ….

    Using MVC frontend pages in Umbraco 4.11.1

    Umbraco CMS

    Umbraco had a rough year, is the least we can say . After working on version 5 from the bottom up for almost 2 years they decided beginning 2012 to stop the version 5 with reason. Although I understand the reasons I’m very glad I didn’t had put any effort yet in the V5.

    Umbraco is an Open Source ASP.NET Content Management System that gives the developer a lot of freedom without interfering with the generated HTML, CSS and javascript. It has a steep learning curve but when you start to find your way into it, it gives you a lot of freedom.

    At my company we used the 4.7 version to create the company websites (abc-groep.be). In that version we could only use Webforms user controls to incorporate our own business logic. As I’m a big fan of MVC this stopped me to use the CMS for personal projects. The V5 version was the big promise to use MVC in Umbraco (it was actually build on MVC). Lucky for me the Umbraco developers have taken into account that MVC is a big deal for developers and while the V5 was dead the V4 version branch was enriched with MVC.

    In this post we’ll look into how you can use your own MVC controllers and views to create front end pages in an Umbraco installation.

    Installing Umbraco in new web project

    Let’s startup Visual Studio and create a new blank web project. Give it a name (here I used UmbracoMVCDemo) and click create. You’ll see that VS only creates an empty project with just 1 file, the web.config.

    image

    image

    After creating the project we’ll add Umbraco by Nuget. Open the Package Console Manager and type “PM> Install-Package UmbracoCms” and hit enter. You’ll see the installation of a couple of dependencies passing by and you should see “Successfully added ‘UmbracoCms 4.11.1’ to UmbracoMVCDemo.”. If you hit F5 you’re browser should fire up  and show the installation page of your local Umbraco CMS.

    image

    Folow the instructions to create your database (MS SQL Server, MS SQL Express Edition, MYSQL and) Microsoft SQL CE 4 database. Choose an administrator user and choose to have no template installed. (if you want you can install these later on).

    After installation go to http://localhost:53998/umbraco/login.aspx (choose the correct port as configured in IIS Express or IIS). Log in with your just created administrator and you should see an empty website.

    image

    We then only have to specify we want to use MVC for our views. Open up the Umbraco.Settings.config file in the config folder under the root. Find and alter the next setting:

    <defaultRenderingEngine>MVC</defaultRenderingEngine>

    Adding new project

    We start by adding a new project in our solution with a right click on the solution and choose Add – New Project

    image

    Choose a new ASP.NET MVC 3 Web Application and give it a name (in this demo UmbracoMVCDemo.FrontEnd).

    image

    In the next dialog choose for an internet application with Razor as View engine. We will need to have the Umbraco core libraries that we can add with Nuget. Open up the Package manager Console and type “Install-Package UmbracoCms.Core”, hit enter and wait for the “Successfully added ‘UmbracoCms.Core 4.11.1’ to UmbracoMVCDemo.FrontEnd.” message.

    Create new controller

    Add a new controller by right clicking the controllers folder and choose for Add – New Controller. Name it DemoController and choose for the Empty controller template.

    The controller needs to inhered from the Umbraco.Web.Mvc.RenderMvcController controller instead of the default ASP.NET Controller class. Next to that we’ll have to override the default Index Action method. We need to add a view for the Index action and add some test text in the view.

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Web;
    using System.Web.Mvc;
    using Umbraco.Web.Models;
    
    namespace UmbracoMVCDemo.FrontEnd.Controllers
    {
        public class DemoController : Umbraco.Web.Mvc.RenderMvcController
        {
            //
            // GET: /Demo/
    
            public override ActionResult Index(RenderModel model)
            {
                //Do some stuff here, then return the base method
                return base.Index(model);
            }
    
        }
    }
    
    
    @{
        ViewBag.Title = "Index";
    }</pre>
    <h2>Index</h2>
    <pre>
    This is our own Demo controller and View.
    

    And that’s it, we’re ready to build. After building our Frontend project we’ll have to copy the generated UmbracoMVCDemo.FrontEnd.dll from the bin folder to the bin folder of our Umbraco installation. Next to the DLL we have to copy the Index.cshtml view from the views/Demo directory to the Umbraco installation views subfolder.

    Configuring Umbraco for our DemoController

    Umbraco uses a technique called ‘Hijacking routes’ to allow custom controllers. Basically will Umbraco for every Document Type check if he can find a controller that inhered the MVCRenderController and has the same name as the document type. If they can find a controller they will execute the overridden Index Action.

    Open up your Umbraco interface in a browser window and click on settings. Right click on Document Types and choose Create. Type Demo and uncheck “Create matching template”. That’s all configuration that has to be done.

    Go to the content section and right click on Content and choose Create. Select the Demo document type and name it Demo. After the entry is created click on the ‘save and publish’ button on top of the entry. Open up a  new browser tab and go to http://localhost:53998/Demo (change the port to the port in your IIS or IIS Express settings) and you’ll see our test text appearing.

    image

    Using own models, …

    At our.umbraco.org you’ll can find a lot of information on how to use your own custom models, bypass the Umbraco pipeline with your own routes, use templates to access different actions, ….

    Automate the copy process

    The copy action of the dll and your views can get a bit tedious after some time (in my case after the first time) but you can use after build events to automate this process. Open up the properties of the FrontEnd project and select the Buils events tab.

    In the Post-build event command line add:

    xcopy $(TargetPath) $(SolutionDir)UmbracoMVCDemo\bin\ /C /Y
    xcopy $(ProjectDir)Views  $(SolutionDir)UmbracoMVCDemo\Views\ /E /Y
    

    The views location

    We saw before that we had to copy the Index.cshtml view from our Frontend project to our Umbraco installation root – Views subfolder. Especially if you want to use multiple actions in a controller the Umbraco Views folder gets messy. I’m more a fan of a folder per controller with in there the views for that controller.

    That’s possible but there is a catch. The Umbraco engine checks if the view is present in the Views folder before allowing the request to use your controller. If the view is missing an empty page is returned. But if we add a Demo folder and add our index.cshtml view in there we’ll see that that view will be used before the view in the root views folder. (This seems to be intended and default ASP.NET MVC behavior)

    Therefor we only need a placeholder view in the basic views folder. This can be an empty view or just add a text that refers to the correct folder. And we can then create a Demo subfolder and add the correct views in that Demo folder.

    The post build action will also copy the views to the correct subfolder instead of the root views folder.

    Second part to be careful with is when creating multiple controllers there will be multiple Index.cshtml views what will give problems in the root views folder. I can strongly recommend creating a second action with an unique name so you can use templates with a unique name for every controller.

    Source

    You can find the demo project at Github.

    Next…

    In a next blog post we’ll look into using our own MVC controllers to create backend functionality.