Dictionary Index Lookup Vs Contains Key Vs List Contains Vs Linq… Speed Test/Texas Tornado Match

Ok so with MVC comes the use of Routes which calls in the need to compare request values to see which route to use. Now before I even bother with that headache (Although it’s getting better AND will be a post) I ran into a situation where I would have to check a passed in string against a list of strings to see if it matches any of them.

One thing I like to do is use Dictionaries. They are just plain convenient when it comes to looking things up or matching values to get methods. But what if I don’t really have a value to find with a key? What if finding the key is all that matters? Say I have a list of strings and I just want to know if the list contains that string, sounds like a job for an array or list right? Wouldn’t it be silly to create a dictionary like:

  Dictiontary<String, String> someList = new Dictiontary<String, String>();
  someList.Add("INeedThis", ""); someList.Add("ThisToo", "");

and do this:

  if(someList.ContainsKey("INeedThis"))

If I don’t actually care about the attached value? I’m sure I’m breaking a rule somewhere… but what if it was faster overall? What if ContainsKey is faster than a list using Any, Contains, FirstOrDefault, or where? Turns out it is. Here’s the method I used.

  public void TimeRun(Holder toHold)
  {
    Int32 maxLength = 1000;

    Dictionary<String, String> stringDictionary = Enumerable.Range(0, maxLength).Select(item => RandomTool.RandomString(item)).ToDictionary(item => item, item => item);
    List<String> stringList = stringDictionary.Select(item => item.Key).ToList();

    String chosenString = stringList[RandomTool.RandomInt32(0, maxLength)];

    Stopwatch runTime = new Stopwatch();

    runTime.Start();
    stringDictionary.ContainsKey(chosenString);
    runTime.Stop();
    toHold.DictionaryContainsKeyTime = runTime.ElapsedTicks;
    runTime.Reset();

    runTime.Start();
    String junk = stringDictionary[chosenString];
    runTime.Stop();
    toHold.DictionaryStraightIndexCheck = runTime.ElapsedTicks;
    runTime.Reset();

    runTime.Start();
    Boolean junkThree = stringList.Contains(chosenString);
    runTime.Stop();
    toHold.ListContains = runTime.ElapsedTicks;
    runTime.Reset();

    runTime.Start();
    Boolean junkTwo = stringList.Any(item => item == chosenString);
    runTime.Stop();
    toHold.ListLinqAny = runTime.ElapsedTicks;
    runTime.Reset();

    runTime.Start();
    String junkFour = stringList.First(item => item == chosenString);
    runTime.Stop();
    toHold.ListLinqFirst = runTime.ElapsedTicks;
    runTime.Reset();

    runTime.Start();
    IEnumerable<String> junkFive = stringList.Where(item => item == chosenString);
    if (junkFive.FirstOrDefault() != String.Empty)
    {

    }
    runTime.Stop();
    toHold.ListLinqWhere = runTime.ElapsedTicks;
    runTime.Reset();
  }

Crazy simple, and why shouldn’t it? Am I right? Am I right? Ok. As you can see, I gave all the methods a run and timed them using StopWatch. And then I ran it a given amount of times, 200 in this code but I tried up to 10000 also. (I’ll put the test code at the end) The test was to go through a list of a thousand strings, each string increasing in length. (Yeah I could have done random size strings but I’m lazy)

What did I find out? Well if it didn’t throw an exception, a straight index search on a dictionary is fastest:

someList["INeedThis"]

And pretty consistently fast. Around 2600 ticks or so on average on multiple runs. (so 10 iterations of parent method running 200-10000 interations of the test method) Next fastest was the ContainsKey method on the dictionary, usually around 2-4 times faster than the next in line good old List.Contains. What I did find surprising is that all the Linq methods failed on this one. I figured that once the first run was through, it would be at least as fast as Contains. (Linq always sucks the first time through) Yeah not so much though. Contains was always faster. Sometimes it was close. Sometimes not even. Here are some example runs:

Dictionary_ContainsKey: 15805
Dictionary_StraightIndexCheck: 2926
List_Contains: 34559
List_LinqAny: 96575
List_LinqFirst: 56541
List_LinqWhere: 64678 

Dictionary_ContainsKey: 7264
Dictionary_StraightIndexCheck: 2676
List_Contains: 29970
List_LinqAny: 41280
List_LinqFirst: 58313
List_LinqWhere: 45669 

Dictionary_ContainsKey: 6773
Dictionary_StraightIndexCheck: 2636
List_Contains: 32366
List_LinqAny: 38670
List_LinqFirst: 33859
List_LinqWhere: 41288

All in ticks. Now mind you, none of these are horribly slow so it probably just comes down to reability and ease of understanding. Personally I like the Dictionary way, so at least speed wise I’m on track. As for looks? That’s a personal thing.

Rest of the code. Here is the parent method. This is a unit test hense the .Assert but it could easily be adapted to any output.

  [TestMethod]
  public void RunTime()
  {
    Int64 overallDictionaryContainsKeyTime = 0;
    Int64 overallDictionaryStraightIndexCheck = 0;
    Int64 overallListContains = 0;
    Int64 overallListLinqAny = 0;
    Int64 overallListLinqFirst = 0;
    Int64 overallListLinqWhere = 0;
    Int32 loopMax = 200;

    for (Int32 loopCounter = 0; loopCounter < loopMax; loopCounter++)
    {
      Holder currentHolder = new Holder();

      TimeRun(currentHolder);
      overallDictionaryContainsKeyTime += currentHolder.DictionaryContainsKeyTime;
      overallDictionaryStraightIndexCheck += currentHolder.DictionaryStraightIndexCheck;
      overallListContains += currentHolder.ListContains;
      overallListLinqAny += currentHolder.ListLinqAny;
      overallListLinqFirst += currentHolder.ListLinqFirst;
      overallListLinqWhere += currentHolder.ListLinqWhere;
    }

    Assert.IsTrue
    (
      false,
      " Dictionary_ContainsKey: " + (overallDictionaryContainsKeyTime / loopMax) +
      " Dictionary_StraightIndexCheck: " + (overallDictionaryStraightIndexCheck / loopMax) +
      " List_Contains: " + (overallListContains / loopMax) +
      " List_LinqAny: " + (overallListLinqAny / loopMax) +
      " List_LinqFirst: " + (overallListLinqFirst / loopMax) +
      " List_LinqWhere: " + (overallListLinqWhere / loopMax)
    );
  }

And the holder class which is a nothing class. I just didn’t care for having to add parameters to the child mehod.

  public class Holder
  {
    public Int64DictionaryContainsKeyTime { get; set; }
    public Int64DictionaryStraightIndexCheck { get; set; }
    public Int64ListLinqAny { get; set; }
    public Int64ListContains { get; set; }
    public Int64ListLinqFirst { get; set; }
    public Int64ListLinqWhere { get; set; }
  }

Couple Notes:

  • StopWatch is in System.Diagnostics
  • RandomTool is actual a class of mine. Nothing special about it. Just makes a string of X length with all random letters.
  • This can not be rebroadcast or retransmitted without the express written permission of my mom.

ASP.Net MVC: Is the Use of jQuery/Asynchronous calls a Must?

In this wonderpost, I questioned the use of jQuery in the mind of the purist MVCist. (If that’s not a word yet, it is now and I expect royalties.) Now I’ll flip it tool style and ask it from another direction: Can MVC survive without jQuery?

Here’s the situation: I have a site that I made a while back (Basically a phone friendly chat room emulator) that was kind of experiment on a couple levels, one being the use of Entity Framework and WebForms. (It’s here if you are curious… right now you have to make an account though. Never got around to allowing non logged in users.) Another part was to see if I could make a multi phone compliant page that had to be fast and work on as many phones as possible. (Anything with a browser at least) And had to be touch friendly. (No links, just buttons)

A week back I decided, after trying to make new sites with MVC, to take an existing work and really give MVC a rugged test. Eythere was just that site. Now personally I thought this would be pretty simple since MVC seemed really good at simple, non business sites. What could be more simple than a chat room emulating site that has no javascript? (Aside from whatever Javascript WebForms use for postbacks) I mean in the end, it’s just a bunch of buttons right?

Yeah turns out what those buttons did was at the heart of my issues. You see, with WebForms, everything done on a page is meant for that page. Sure you can defer how things are handled from the page to another tier, but the action itself belongs to the page. For example, say that a user wanted to make a room a favorite, well there’s a button named favorite that he/she clicks to do so.

 Oh the lovely colors.

In webforms, the Room.aspx page would handle the Favorite button click and the current user’s favorite list would have the room removed or added depending. But the call to the add/remove would be handled by the button’s click event and then the page would be reloaded. Something we in the know call “easy”. (Like that word? Use it. It’s free because I’m such a nice guy)

Now with MVC it’s a whole new ballpark. With controllers, it becomes of question of what controller should handle this. It could go onto the room controller, but in the end it deals with a user and adding something to the user. So it’s fair to say that this is probably a job for Superman if Superman happens to be a UserController or you’ll most likely end up repeating code.

Now the issue: How is this handled? Whether link or button, something has to be posted to the user controller. Posting to the controller means that the controller, on finish, has to create a model and send it to a view. Well this kind of sucks doesn’t it? The user clicks the + Favorite button and bam is taken to a whole new page.

SURPRISE IT’S BEEN ADDED TO YOUR FAVORITES!

Not what I would call fluid. The other thought it to have a redirect, which in the end is more fluid in that it takes the user back to the original page BUT with that you have to send a lot of information back and forth. After all, in MVC a lot of the information for a given “page” is passed using get. So for instance you have a room with a roomid and a page number and a count of items to show and possibly a sort you’re looking at:

/Room/1/?pageNumber=1&amountToShow=5&sort=Date

Which means all of that has to be sent to the User controller in order to save the state, has to be put in session/some kind of user cache, or you just send the entire url. In any case, something has to be tossed around for this to work. More seamless than the other way but it is a lot of work for something so simple.

When it comes to ease of UI design, jQuery/asynchronous operations are a must.

Something I think that the MVC idea fails at it in a UI sense it was makes it strong in a programming sense, logical separation of tasks. Right off the bat, I hated the way WebForms handled AJAX “web methods”. Having to repeat methods per page so they could be “registered” was ugly and a pain. MVC and it’s separation/REST ways makes asynchronous calls so easy. However, what used to be a simple operation in WebForms has now become cumbersome without outside help. Straight up, it seems impossible to do non drill down design without using jQuery or some kind of javascript library equivalent without killing the separation that MVC seems to embrace.

Why is this a problem? For the most part it isn’t. Most people aren’t going to try something like EyThere (Wouldn’t recommend it, it was a pain) since how many people create sites with multiple phones in mind? However, it does serve to show what seems to be a glaring annoyance with MVC. Either use asynchronous calls or just sink.

ASP.Net MVC Issue: Form With Get Method Not Passing Request Values

DISCLAIMER: This has a solution but there has to be a better way.

So I came across a really odd situation with the MVC framework today, and I’m on the edge of some serious Nerd Rage. Here’s the issue:

I have a route that looks like this:

  "{controller}/{action}/{roomId}/{name}"

And a form that looks like this:

  <form action="Room/Index/6/SomeThing?pageNumber=1&amountToShow=5" method="get">
   <button type="submit">SomeButton</button>
  </form>

Can’t get much simpler… or is it more simple? Whatever. Point is, when the the button is clicked, it should go to that page like the form action says to. And it does, sort of. For some reason it ends up stripping the values from the pageNumber and amoutToShow request variables. How do I know this? Well break points at certain times show me that the current ActionExecutingContext ActionParameters (the thing that stores all the request parameters) has the two I need but neither has a value. And since they are not included in that route description, it must see them in the url.

You might think it has to do with forms and submitting, and it might except here’s the kicker… with a route like this:

   "{controller}/{action}/{roomId}/{name}/{pageNumber}/{amountToShow}"

All of a sudden it can see the values. Mind you the url ends up like:

  Room/Index/1/Something/1/5

Which you would expect from the route definition. To add to the fun, if I add hidden values:

  <form action="Room/Index/6/SomeThing?pageNumber=1&amountToShow=5" method="get">
    <input type="hidden" name="pageNumber" value="1" />
    <input type="hidden" name="amountToShow" value="5" />
    <button type="submit">SomeButton</button>
  </form>

It works with the original route. Doesn’t really make sense to me, but my understanding of the system isn’t exactly expert level. However, I could buy this if the hidden field method worked and the extended route definition didn’t.

For now I can use the hidden input method since there’s no issue with showing the parameters. Just not what I’d call “Best solution” types stuff.

Sigh. MVC : One step forward, two steps back.

Entity Framework: Am I Just Being Stubborn?

Before I started using The EF, I had a couple years of use with LLBLGEN and about a year into NHibernate. I started dabbling with Linq to Sql for a bit but found it a bit lacking (As it was new, go figure) and decided to go forward with Entity Framework due to the fact that I really like Linq, and although LLBLGEN had Linq integration, I figured the Entity Framework would be the first out there with any changes in Linq. (Not exactly sage like foresight) And at the time HQL was still the thing for NHibernate, and although it’s not a bad design, it’s a nightmare waiting to happen since you can’t get a compile time check on the HQL strings. So it seemed like the right idea at the time, and despite what I’m about to write, it may still be.

I saw a lot of people bagging on it saying that it was missing this feature or that feature. Most people were annoyed that there wasn’t built in lazy loading. Some were annoyed with the fact that the classes are coupled to persistence. Others just complained because it wasn’t [insert your ORM here]. I on the other hand championed the cause with this in mind:

It’s just a framework. The tools are there do what you need, you just need to build it.

Take as is, the word framework is just that. It’s just a start. It’s that point where a house is just a bunch of wood nailed togther. You can’t really live in it as is… well I suppose you could?… but it has what you need to produce a complete house. Just takes a little time and effort. You can already see where this is going. After all, I’ve already figured out I like pain.

So the first issue I ran into was the context itself. You see, I was already used to multiple calls to the database being done with one “connection” and by that I mean not this every time I wanted to get something:

  using(var entity as ToolEntities)
  {
    entity.Tools.Select(...)
  }

Mind you my syntax is more psuedo in that example, but the idea was simple; It was going to be annoying to have to do that every time I wanted something from the database. So after hours of thought and some research, I came up with a solution that would open one “connection” per request that all queries within that request would use. Ok so that was out of the way, and I can’t compain since after all:

It’s just a framework. The tools are there do what you need, you just need to build it.

The next thing I ran into was lazy loading a many to one or one to one property. I had already seen stuff on many to many or one to many loading. Simple method .Load(). To my surprise there wasn’t such a method for this (Not directly at least) so I was stumped. Time to go back to LLBLGEN? Hell no, mush on. And mush on I did. I found the .Include method:

  context.Tools.Include("Favorites")

Boo yah, and I’m back in the game… or was for a little bit until I realized that this is pretty bad since Include loads the whole object you are including. This could be ridiculous if you have an object with 10… 100… ONE MILLION many to one or one to one properties. So once again, hours of thought and research later I came up with a solution that was eh good enough but wasn’t exactly what I’d hoped for.

It’s just a framework. The tools are there do what you need, you just need to build it.

Another issue I ran into was a creating an IN clause method. Contains just doesn’t work with Entity Framework. Straight up, flat out, no can do sir. So again, with time and research I found a way to do this (Which I haven’t posted yet due to the fact I didn’t come up with the solution and I can’t find the link I got it from). Of course there was the order by situation, oh and this gem, and this that I can only describe as a god damned mess. Really? It’s in the context yet you still have to run a query on it? Really??

I’m not going to quit on Entity Framework just yet but after a little introspection I have to admit that I’m close to responding to my own righteous:

It’s just a framework. The tools are there do what you need, you just need to build it.

With a big :

Go f- yourself.

This whole situation reminds me of a saying:

If you’re in a poker game and you can’t figure out who the patsy is, you’re the patsy.

It doesn’t take much imagination to apply that saying to my situation and to be honest I might be falling into the same situation with the MVC framework. Time will only tell with that. Although at least with MVC I have plenty of patsies with me. I think with Entity Framework, I might be the only one losing money.

jQuery Slide and Pairing a Div’s Hover With a Div’s Slide

So this could be filed under “I did it because I can” (which is a really good mantra in science…) and am not sure I’ll use it but it is useful in the concept.

LIVE EXAMPLE HERE

The idea is to set the hover on one div to show/hide another div WITHOUT having to use some kind of id/name trick (as in setting the id to “divHide1”) and to have it be completely self setting in the .ready method. Why would this be at all useful? Say you want to roll through a list of things and generate the divs, but want to defer the jQuery until the page has loaded. And you don’t want to have to resort to:

    <div id="someDiv<%= someId %>"></div>

like structure where you parse some identifier from the id property. Mostly because you have no idea how many someDiv1s there could be on the page. It could be a highly reused name (someDiv) and that could lead to a mess.

Also the reason ‘cuz. If you have any more questions of why after that answer, well you’re just being annoying.

Anywho, here’s the actual html for structure.

   <div class="mainDiv">
      <div class="showDiv" id="oneTop">
        Kaneda?
      </div>
      <div class="slideDiv" id="one">
        What do you see?!
      </div>
    </div>
    <div class="clear"></div>

Now I get that this isn’t off a foreach, but it doesn’t take much to figure out that it’s easily made into a loop if you just loop it over and over. Why? Because first there are no ids or names so that you can’t have two of the same name and also because that chunk is self contained.

So what is going on there? Simple, you have a parent container that holds a div to hold and a div to hover over and the other that will be shown/hidden.

Here’s the styles involved just incase you care:

    <style type="text/css">
        .clear
        {
        	clear:both;
        }

        .leftDiv
        {
        	float:left;
        }

        .mainDiv
        {
        	margin:5px;
        	height:200px;
        }

        .rightDiv
        {
        	float:left;
        }

        .showDiv
        {
        	float:left;
        	margin-right:5px;
        	height:200px;
        	background-color:Silver;
        }

        .slideDiv
        {
        	background-color:Teal;
        	height:200px;
        	position:absolute;
        	float:left;
        	z-index:100;
        }
    </style>

That doesn’t entirely matter, but it does to show the div “sliding” over another div. Kind of a little somethin’ somethin’ I threw in at no extra cost. Now for the jQuery:

First the method for setting the mouse over and out events for the show div, we turn to .hover:

    function setHover(currentSlideDiv, currentStableDiv)
    {
      currentStableDiv.hover
      (
        //first parameter is the method for showing the div on mouse over
        function()
        {
          if (currentSlideDiv.is(":hidden"))
          {
            currentSlideDiv.show("slide", { direction: "left" }, 100);
          }
        },
        //second parameter is the method for hiding the div on mouse out
        function()
        {
          if (currentSlideDiv.is(":visible"))
          {
            currentSlideDiv.hide("slide", { direction: "left" }, 100);
          }
        }
      );
    };

Now for the method that actually uses these:

    //This is used to take in one of the main divs and set all the
    //show and slide divs within.
    function setChildrenDivs(mainDiv)
    {
      //get all the show and slide dvis within the main div
      var mainChildrenStableDiv = jQuery(mainDiv).children(".showDiv");
      var mainChildrenSlide = jQuery(mainDiv).children(".slideDiv");

      //loop through the list of show divs
      for (var loopCounter = 0; loopCounter < mainChildrenStableDiv.length; loopCounter++)
      {
        //Get the show div and it's corresponding slide div using the
        //two lists and the current counter.
        var currentStableDiv = jQuery(mainChildrenStableDiv[loopCounter]);
        var currentSlideDiv = jQuery(mainChildrenSlide[loopCounter]);

         //This is to make sure the slide is where it should be.
         //to the right of the show div.
         var containerPosition = jQuery(currentStableDiv).position();
         jQuery(currentSlideDiv).css({ left: containerPosition.left + currentStableDiv.width() });
        //Set the mouse over and the mouse out on the show div
        setHover(currentSlideDiv, currentStableDiv);
      }
    }

Now the final touch, the .ready method:

    jQuery(document).ready
    (
      function()
      {
        //hide all the slide divs
        jQuery(".slideDiv").hide();

        //find all the parent divs
        var mainDivs = jQuery(".mainDiv");

        //set the children
        for (var loopCounter = 0; loopCounter < mainDivs.length; loopCounter++)
        {
          setChildrenDivs(mainDivs[loopCounter]);
        }
      }
    );

And boom, now you have multiple show/hide (Slide in this instance). Now if I could just find a use for it…

Oh yeah and you’ll need jQuery 1.3.2 and jQuery ui 1.7.2 to use this. At least those are the versions I know this works with.

Update: Due to popular demand… one person… Source can be found here.

Are jQuery and Asynchronous Calls against MVC?

This is something I started to think about the other day as I have been diving deeper into MVC. In it’s purest form, at least to what I understand, is that the whole idea of MVC is to have a separation between information handling (controller), information delivery method (model), and the information presentation (View). What this means is that the View itself should only be around to display or return information, but that’s it. Have a form? Great, here’s the information you need to fill out and I’ll send it on it’s way. And even with that idea, the View itself doesn’t really send anything. It just passes what it knows off to the model and it’s back in happy land. (The view gets nervous sometimes… performance anxiety) The whole system works well in actuallity, it does what is expected. There are no events simulated and possibly tangled. There’s no smoke and mirrors when it comes to the display and what the user is seeing. I want a grid of users? Here it is. I want to edit this user? Great! I’ll take you to the page that does that. Life is great and wonderful.

But wait, here comes jQuery, Json, and Asynchronous Calls. Now I can do crazy things with the view that I could never do before. Remember that grid of users? Well you don’t really have to go to another page to edit. Hell, just click this link and a form can appear to take in what you need to change. Even better, the grid itself will change and allow you to edit any row you want, and when you’re done the grid will come back all new an refreshed. What a crazy party this is now. Kind of sounds familiar right? Yeah WebForms. With these three helpers we’ve now cut out the middle man in the Model and passed the savings onto you. Essentially the whole idea has been chucked and jQuery has allowed us to invoke the same event model WebForms was chastised for. In fact, you could go to the logical end and have one page once again that controls everything. The View once again has the power much like it did in WebForms (provided you look at the Page and code behind being more like a View than separate entities). Isn’t that the whole thing we were trying to get away from?

I suppose you could make the argument that no one is putting a gun to your head to use jQuery for anything other that simple things like UI manipulation (date pickers, sliding controls, drag drop, ect) but that wouldn’t be the situation being argued here, now would it?

ASP.Net MVC: Create a link method… ie JUST GIVE ME THE STUPID URL

One thing that kind of annoys me is the situation where you just want a url, but you don’t want one of the prepackaged links using:

   Html.RouteLink
Or
   Html.ActionLink

Mostly because you want to be able to create your own html and you only want the created url. Well turns out there’s a method for this: The RouteUrl method on the UrlHelper class. Down side is that it’s not static and takes a few lines to use, so not cool UI design side. Well here’s a method that uses that method and gives you a method to exact a method of victory. I think that sentence had promise, but fell short of complete failure.

Anyways, here it is… The “Just give me the stupid url” method, CreateUrl for short.

        public static String CreateUrl
        (
          RequestContext context,
          RouteCollection routeCollection,
          String routeName,
          String controller,
          String action,
          RouteValueDictionary routeValues
        )
        {
            //Create the helper
            UrlHelper neededHelper = new UrlHelper(context, routeCollection);

            //get the route to check what it is holding at far
            //as defaults go
            var neededRoute = (Route)routeCollection[routeName];

            //this might be overkill honestly.  Basically in case the
            //Route contains the "controller" key only then add it to the
            //values for the route.  Otherwise just ignore.  It's possible
            //someone might pass in a controller/action but the route
            //doesn't take them. At which point you'll
            //be showing the "Aw maaaaan" face.

            if (!String.IsNullOrEmpty(controller) && neededRoute.Defaults.ContainsKey("controller"))
            {
                routeValues.Add("controller", controller);
            }

            if (!String.IsNullOrEmpty(action) && neededRoute.Defaults.ContainsKey("action"))
            {
                routeValues.Add("action", action);
            }

            //And then the call to create the url string.
            return neededHelper.RouteUrl(routeName, routeValues);
        }

And in use View side:

  <a href="
  <%
  CreateUrl
  (
    Html.ViewContext.RequestContext,
    Html.RouteCollection,
    GeneralConstants.RouteDefault,
    "SomeController",
    "SomeAction",
    new RouteValueDictionary { { "id", 1 } }
  )
  %>"
  >
  Something is linked
  </a>

Now mind you I did use a link there so it seems like a silly example. However, you can do many other things now that you just have the url itself.

Entity Framework: LINQ to Entities only supports casting Entity Data Model primitive types

So in typical tool fashion I posted this little gem without realizing a glaring error… the order by clause. The whole idea is to create a method that can get a collection, sort it, then grab a certain number for paging. The issue was this:

    Expression<Func<K, IComparable>> orderBy

The problem comes in when the entire expression is run, meaning when Entity Frame work takes:

    context.Select(selectMethod).Where(whereClause).OrderBy(orderBy).ToList();

and creates SQL out of it. Entity Framework doesn’t really know how to handle IComparable as it has not primitive/sql type to match to it. Why it can’t see the underlying type of say DateTime, no idea, but this is life.

So this should be an easy fix, right? Eh… yeah. First I thought instead of IComparable I could just convert to some kind of primitive type so that the EF could be on it’s merry. Not so much. Turns out this probably isn’t possible.

Well thanks to a push from Ben M at The O Flow I got to thinking about how to attack this. Instead of sending in an expression for the order by, why not send in a method that would take in a query and tell it how to order itself. Sounds hard right? (If not then you’re obviously too cool for this school) Well it’s not, just a different way to think about it.

Right off the bat, the change to the method signature would look like this:

Old order by parameter signature:

    Expression<Func<K, IComparable>> orderBy

New order by parameter signature:

    Func<IQueryable<T>, IOrderedQueryable<T>> orderBy

So what does that mean? It means that I am going to supply the method with a method that will take in a query and return an ordered query… well not ordered yet per se but the blueprint on how to order when that time comes around. Now here’s how that’s used:

First you need the query:

    var initialQuery = query
    .Where
    (
        somethingEqualsSomething
    );

Then you apply the orderby method, and in the case of the original paging method, the paging too:

    var orderedQuery = orderBy(initialQuery);

    returnValue = orderedQuery
      .Skip(numberToShow * realPage)
      .Take(numberToShow)
      .ToList();

So overall, doesn’t look too much different. Just instead of supplying the OrderBy method with a Func, you give a method that creates an ordered query.

How would you use this? Remember the signature was (whereClause, selectClause, orderBy, pageNumber, numberToShow, realPage, totalCountOfPages)

    Tools.GetListForGrid
    (
      tool => tool.EntityId == userId,
      tool => new { Name = tool.Name },  //Select Clause, I'll get to that next
      toolOuter => toolOuter.OrderBy(toolInner => toolInner .Name),  //OrderBy
      ...
    )

Two things you might notice. One would be the OrderBy signature:

   toolOuter => toolOuter.OrderBy(toolInner => toolInner .Name),

What the hell? Remember the method you are sending takes in a query and returns an ordered query. toolOuter is your query, toolOuter.OrderBy(toolInner => toolInner .Name) is your blueprint (IOrderedQueryable) on how it should be queried.

Second thing is that when I showed how to use the OrderBy method above:

    var orderedQuery = orderBy(initialQuery);

    returnValue = orderedQuery
      .Skip(numberToShow * realPage)
      .Take(numberToShow)
      .ToList();

I didn’t include the select clause. Partially because I didn’t want to complicate it yet, partially because it has it’s own issue. If you’re like me, you use the select clause a lot. Why? Because it limits the amount of information you take from the database. Say if you are trying to fill a drop down list, why select an entire Tool object (Which could have a ton of mapped properties) when all you need is Id and Name? Seems silly. That’s where the Select clause comes in. Now the issue is where to put the order by. You would think after the select clause, since you want to sort on only what you are selecting. Problem is, with paging that gets screwed up. The way sql server “pages” is that it selects twice.

Select all the things that match this where clause.
Select a certain number of items from that starting at X.

The first select creates a dataset with row numbers, and the second one selects the items based on those row numbers. IE take 5 starting at row 5. Now the way the EF handles the order by is to grab the info you need from the Select (Ordered by something… you don’t get to choose) and THEN order and grab. As you can guess this may not give the needed results. So how can this be solved? Order before the select as witnessed in the new and improved method:

public static IList<K> GetListForGrid<T, K>
(
    this ObjectQuery<T> query,
    Expression<Func<T, Boolean>> somethingEqualsSomething,
    Expression<Func<T, K>> selectClause,
    Func<IQueryable<T>, IOrderedQueryable<T>> orderBy,
    Int32 pageNumber,
    Int32 numberToShow,
    out Int32 realPage,
    out Int32 totalCountOfPages
)
{
    IList<K> returnValue;

    Int32 totalItemCount =
        query
        .Count
        (
          somethingEqualsSomething
        );

    totalCountOfPages = Methods.TotalCountOfPages(totalItemCount, numberToShow);
    realPage = Methods.GetRealPage(totalCountOfPages, pageNumber);

    var initialQuery =
      query
      .Where
      (
        somethingEqualsSomething
      );

   var orderedQuery = orderBy(initialQuery);

    returnValue = orderedQuery
      .Select
      (
        selectClause
      )
      .Skip(numberToShow * realPage)
      .Take(numberToShow)
      .ToList();

      return returnValue;
}

And usage:

    Tools.GetListForGrid
    (
       tool => tool.Id == userId,
       tool => new { Name = tool.Name },  //Select Clause
       toolOuter => toolOuter.OrderBy(toolInner => toolInner .Name),  //OrderBy
       pageNumber,
       amountToShow,
       out realPage,
       out totalCountOfPages
    );

Now this one actually works with Entity Framework and not just Linq to objects like the last one.

Asp.net Mvc Master Page: System.Data.Entity Must Be Referenced

Ok maybe you’re better at figuring out the obvious than I am, but this was annoying me. On my master page I happened to be using an object from another assembly that used the Entity Framework. Now by habit, I have this assembly:

System.Data.Entity, Version=3.5.0.0

Referenced to the UI project, so it was a little of a surpise to me that I was getting this error on the master page. (And the views, but I don’t use anything in Views not supplied by Models) I was at a loss as to how to get the thing to work.

Now I could use this:

<%@ Assembly Name=”System.Data.Entity, Version=3.5.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089%>

On the page, but would be silly to do that all the thing, and DUH in WebForms… or ASP.Net Classic, is to put it in the stupid web config.

  <system.web>
    ...
    <assemblies>
      ...
      <add assembly="System.Data.Entity, Version=3.5.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089"/>
      ...
    </assemblies>
  ...

No more problems.

Entity Framework: Reusable Paging Method

OBSOLETE… sort of. Found an issue with this and am posting the issue today. (07/21/2009)

A while back I decided to show some paging stuff and I forgot that recently, well ok in the last few months, I actually put together a sort of generic extension method to take care of most of the paging situations I was running into. Reason being is I got tired of writing the same method over and over. I figured I’d share it in hopes that somehow, somewhere, it would keep someone’s dream alive. That and I figured I needed to make up for not posting for a while… which to some of you might have been a good thing.

Anywho, it’s not to difficult actually. Small note though, when reading the code you will see the TotalCountOfPages method and the GetRealPage method. If you can’t figure out what those are then I just don’t know what to do. (HINT: They are linked somewhere in this very post! Side note: This post is on track to match the normal self linking per post quota on atypical Codding Horror post)

public static IList<K> GetListForGrid<T, K>
(
  this ObjectQuery<T> query,  //Extension method syntax... basically takes in any SomeEntities.SomeTableRepresentation
  Expression<Func<T, Boolean>> somethingEqualsSomething,  //Where clause
  Expression<Func<T, K>> selectClause,
  Expression<Func<K, IComparable>> orderBy,  //Turns out order by just needs anything that is IComparable
  Int32 pageNumber,
  Int32 numberToShow,
  out Int32 realPage,
  out Int32 totalCountOfPages
)
{
   IList<K> returnValue;
   //Get the total count of the items.  Need this to do proper paging.
   Int32 totalItemCount =
      query
      .Count
      (
         somethingEqualsSomething
      );

   totalCountOfPages = TotalCountOfPages(totalItemCount, numberToShow);
   realPage = GetRealPage(totalCountOfPages, pageNumber);

   returnValue = query
   .Where
   (
      somethingEqualsSomething
   )
   .Select
   (
      selectClause
   )
   .OrderBy(orderBy)
   .Skip(numberToShow * realPage)
   .Take(numberToShow)
   .ToList();

   return returnValue;
}

As you can see there are a lot of expressions going on and that’s the way this works. It takes care of the annoying stuff that you would have to copy an paste while you supply it the stuff that changes like what you want to select and what the needed items have to match to be selected. Even has something for an order by. What the usage look like?

  ToolEntities.Tools.GetListForGrid
  (
    something => something.SomethingId == passedInSomethingId,  //Where clause
    found => found,  //Select: Simple here.  Usually I use classes to hold info... next post
    found => found.Name,  //Order by
    pageNumber,
    amountToShow,
    out realPage,
    out totalCountOfPages
  );

Really simple, eh?

Why totalCountOfPages and RealPage out? Well the whole idea is that this infomation is valuable to the picker control you will be constructing outside this method call. RealPage is nice for showing what page the pager is on and how to figure out what the previous and next pages will be. TotalCountOfPages is useful for the last page possible. Now the nice thing about the RealPage method is that it won’t give you a non existent page. It will only give you the best possible page if the pageNumber parameter is over the possible amount of pages.

Also do notice that this is an extention method meant to be used with the Entity Framework’s built in system of handling the Table representations say ToolEntities.Tools.(Method will show up here in intellisense). This makes it cleaner to me than having to pass in the actual current context. As in:

GetListForGrid(ToolEntities.Something, …) vs ToolEntites.Something.GetListForGrid(…)

In the end, either way is fine.