Clojure Multi Method Chain with Partial

In my last post I had mentioned that there could be an improvement to
the code below:

(->
  subject-items
  ((partial sort-by :Name))
  ((partial drop (* page-number items-per-page))
  ((partial take items-per-page)))

Simple method chain taking advantage of partial methods. That might
be fine for lesser beings, but not for someone so ruggishly handsome
as me.

To take an easy example that can be run in the REPL:

(->
  (range 20)
  ((partial drop 5))
  ((partial take 5))
  ((partial apply +)))

Create a list of 1 to 20, drop the first 5, take the next 5, and add
them. You know, completely useless in the real world. What irks me
(Yes I used the word irks) is:

((partial ... )).

The reason for the double parenthesis is that as the chain moves to
the next step, it applies the chain result to it. If it were only:

(partial drop 5 [1 2 ... 20])

it would have tried to apply the range list to the partial call.
Uh oh… not good. Now look at it from the correct way:

((partial drop 5) [1 2 ...20])

Now it actually works. Partial is created, and THEN the list is
applied to it. Weeee!

So what’s next? Removing the partial thing. What I want, and you
should want too, is this:

(somemacroname
  (range 20)
  (drop 5)
  (take 5)
  (apply +))

A fair bit less typing, and less words to read. Win and Win.

Now when it comes to macros, It’s a goog idea to remember that macros,
for the most part, is just about assembling lists. After all, Clojure
is a part of the Lisp family. Therefore, there is no difference
between a list, or code.

So to get underway, what do we need? A really cool, and novel name
for the macro.

|->

Next, there needs to be a parameter list for the macro to use. There
are two parts of the original code: The list, and the list
manipulators. Because the list, in this context, is the odd ball, it
gets its own parameter. The manipulators get smashed into one group.
Don’t cry for them. Not them.

(defmacro |-> [original & items]

That way easy. Time to go home.
Ok so that I’m home, I’ll move to the next step:

(defmacro |-> [original & items]
  `(->
     ~original
   )

That the -> an obvious given. The -> is needed. The
~original takes the value in original (range 20), and will resolve it
at expansion time.

Why not just:

(defmacro |-> [original & items]
  `(->
     original
   )

Because if you expand that, the word ‘original’ will just be there,
not the (range 0 20) that it represents.
~original produces:

(->
  (range 0 20)

original (with no ~) produces:

(->
  original

Next step is to figure out how to get this:

  ((partial drop 5))
  ((partial take 5))
  ((partial apply +))

from this:

  (drop 5)
  (take 5)
  (apply +)

Remember, we are creating code by manipulating lists. What we need is
a tranformation from (drop 5) to ((partial drop 5)). How can this be
done? Cons.

(cons `partial %)

Ok, why cons? Every item in the items list is itself a list.

(drop 5)

If “cons” the keyword partial to it I get:

('partial drop 5)

Essentially I am making a new list by attaching partial to the old
one. However, that doesn’t get what is needed.

((partial drop 5))

So guess what? Cons again.

(cons (cons 'partial %) '())

This time I consed the new partial/drop list to an empty list.

(('partial %))

If only there was a way to automate the creation of a list of the
converted items…

(map #(cons (cons 'partial %) '()) items )

WOWOWOW

Now to put it all together:

(defmacro |-> [original & items]
  `(->
    ~original
    ~@(map #(cons (cons 'partial %) '()) items )))

The ~@ is just so that it evalates the map call/result at expansion
time. The @ gets rid of the surrounding parenthesis of the list that
map creates.

Now you do exciting things like:

 (|->
  (range 0 20)
  (drop 5)
  (take 5)
  (map #(+ 1 %)))

AWESOME

Clojure: How to Chain Array Functions Using Partial

Wrote this just recently as a part of way to mock a database query for testing JavaScript controls.

(defn retrieve-subject-items [page per-page sort descending]
  (take items-per-page
        (drop (* page-number items-per-page)
              (sort-by (to-key sort) subject-items))))

Is the “why” important? No. What is important is that all of these list functions have one thing in common:

take =    f(int list)     => list
drop =    f(int list)     => list
sort-by = f(keyword list) => list

The last parameter needed is a list. Mind blown, I know.

This screams for partial functions. What are partial functions? They are functions created to “wrap” existing functions, just with some of the arguments already taken care of. For example:

function drop (numberToDrop listToUse)

Easy enough. It takes in a list, and the number of items needed to be dropped from said list. Now what if you knew that under a certain circumstance, the top 5 will always be dropped. Say there are three lists: MathTestScores, EnglishTestScores, and BiologyTestScores where the “scores” are numbers from 1-100. One rule is no matter which list is used, the top 5 scores will be removed to help with a grading curve. You could do something like this:

drop(5 MathTestScores)
...
drop(5 EnglishTestScores)
...
drop(5 BiologyTestScores)

As you might notice, there is a pattern here. You could create a method that takes in a list, and removes the top 5. Nothing wrong with that.

function drop5(list)

Now, as dumb as this might be, there is some reason you’re told to drop 5 from all lists, and add the sum of each list together. Yes, this example is train wreck from a “makes sense” standpoint. Or it sounds like how to calculate a baseball statistic. Either way: Convoluted. But you are merely the worker. Yours is not to ask why.

You might get something like this:

sum = drop5AndSum(MathTestScores 0)
sum = drop5AndSum(EnglishTestScores sum)
sum = drop5AndSum(BiologyScores sum)

or

sum = 
  drop5AndSum(BiologyTestScores 
    (drop5AndSum EnglishTestScores 
     (drop5AndSum MathTestScores 0)))

Wonderful. Now lets say you use a language that has method chaining. Basically the preceding method sends its result to the following method. Sort of like the example right above this. Only difference is the order is kept.

sum = drop5AndSum(MathTestScores, 0) -> drop5AndSum(EnglishTestScores, *sumFromLastCall*) -> drop5AndSub(BiologyTestScores, *sumFromLastCall*)

Problem with this is that the drop5AndSum function needs two parameters, but it only produces one. So syntactically you’re &$!@ed. This is where partial methods come in. The goal is to get the call to drop5AndSum to only need one argument: the result from the preceding drop5AndSum function.

sum = drop5AndSum(MathTestScores, 0) -> drop5AndSum(EnglishTestScores) -> drop5AndSum(BiologyTestScores)

BUT HOW?!!? Partial methods… and man whorin’.

partialEnglish = dropAndSum5(EnglishTestScores)
sum = drop5AndSum(MathScores, 0) -> partialEnglish

The function the follows “drop5AndSum(MathScores, 0)” no longer needs to have the list passed in. It just needs a number, the sum, and it will do its thing. How is that possible? Look at the original function definition:

drop5AndSum = f(list, int) => int

Now look at partialEnglish

partialEnglish = f(int) => g(list, int) => int

As you can see, partialEnglish takes in the preceding sum result, then applies that to the drop5AndSum function.

Why bother with this? Easier to understand the order of operations.

mathSum = drop5AndSum(MathTestScores);
englishSum = drop5AndSum(EnglishTestScores);
biologySum = drop5AndSum(BiologyTestScores);
sum = 0 -> mathSum(MathTestScores) -> englishSum(EnglishTestScores) -> biologySum(BiologyTestScores)

The intention is a little more readable than the (drop5AndSum … (drop5AndSum … (drop5AndSum …))).

What does this have anything to do with the original block I had at the start:

(defn retrieve-subject-items [page per-page sort descending]
  (take per-page
        (drop (* page 5)
              (sort-by (to-key sort) subject-items))))

Because the order is actually right to left, it may not read well. However, if it read from top to bottom, readability should increase.

(->
  subject-items
  ((partial sort-by :Name))
  ((partial drop (* page-number items-per-page))
  ((partial take items-per-page)))

Now with somewhat meh keywords ignored (Ahem “partial”): It’s pretty obvious what the order of operations is, as is the overall intent. Start with a list, sort it by name, drop a few, take a few. Despite the ugliness of “partial”, it is still far more readable.  Good new is, if you’re using Clojure, you can remove the “partial” part altogether with a macro.

Also if you’re using Clojure, you might notice the double parenthesis.  This has to do with the need to resolve the returned partial method in order to apply it to the passed in chain result.

Clojure: Chaining Lamba Expressions that Act Like Partials

Really simple one. Say you have a chain like this:

(let [username (:username (?retrieve-value result))]
  (let [possible-error-message (text-is-empty username "Username")]
    (add-an-error-message result possible-error-message)))))

Pretty normal, and boring… and not super easy to read. Now this might happen only to someone who is rather new to Clojure (ME), but I figured I’d post it anyhow. Because I am that cool.

Using the -> macro (if you don’t know what that is, look at the Notes below), and lambda expressions (anonymous methods), it can be rewritten to remove a lot of the noise.

 (->
  (retrieve-value result)
  (#(:username %))
  (#(text-is-empty % "Username"))
  (#(add-an-error-message result %)))

That does the exact same thing as the original method, but cleans it up rather nicely. Though it still isn’t super mega ultra hyper readable…dx, it is still much easier to determine what is going on. Now the one thing needing to be noted is the extra () around the lambda expression. Because #(:username %) actually returns the method it creates, the second pair of () is used to call it with the value returned from (retrieve-value result). A little annoying, but really not that big of a deal.

Notes:

The -> macro

-> is used to chain methods together. Say I have two function signatures (PSEUDO CODE ALERT):

  f() :string
  g(string) : string

I could call them like this:

g(f())

With the -> operator I can do this:

f() -> g

As you can see the -> takes the result from “f”, and pushes it into “g”.

The # macro:

This is used to shorthand the normal lambda expression syntax.

(fn [a] (:username a))

Is the same as:

#(:username %)

Somewhat like the => operator in C# is used to replace the old anonymous method syntax from 2.0.

Setting up SqlKorma with Postgresql on Windows 7

SqlKorma is a really nice Clojure DSL for handling things like selects, inserts, but not deletes. Ok, so that’s a lie, it can do deletes too. What would be really useful is if I had a project already setup for all to view. Maybe even something up on github. This is totally not a link to it.

For this walk through I used the lobos_example project as a start since it already had the table creation stuff in it. (This is the walkthrough for Lobos, which is also a walk through for setting up Postgresql.)

First off, if you are using the lobos_example project (if not, this post will walk through the important parts in the komra_example project), there is test that needs to be shanked.  It’s in the test/lobo_example/core_test.clj file:

Also get rid of the old database setting “open-global” in the lobos/config.clj file:

The next step is to add the dependency for SqlKorma to the project.clj file in the project root folder.  Depending on how old this post is, the version may not be correct.

Now it’s time to create the entities that will match the already existing tables.  If you’ve never used an ORM before, this is simply spelling out to SqlKorma what the tables are, their columns, and their relationships.  However, this is not done with any SQL statements.  In fact, the point of this file, and SqlKorma in general, is to create a high level representation of the persistence layer.  Whoa, that got serious fast.  Basically, you are using class like entities so that you can easily query, or persist stuff, without caring about anything database wise.

So anywho, here is the entity file:

A quick note to those new to Clojure, like uuuuhhh me, :use statements can be consolidated to the root namespace.  So instead of:

[korma.core]

[korma.db]

You can do this:

One other note is the example of a relationship.  The symbol “belongs-to” is needed if one entity is owned by another. This being a many to one relationship between many posts, and one user:

One file to make.  Basically it’s a test file that is run to make sure that an insert, select, and delete work:

Nothing all that complicated.  As you can see, there is an insert, assertion (is (empty?….), and a delete.

One other note for those like me who are just picking up Clojure, there are two :use/:require keywords of note in this other note…note.  If you look at the :require block you’ll see the :as keyword.  This is basically an alias to help with keeping typing down.  There might be other reasons, but I’m not the one to ask.  So don’t.  Not kidding.

The second keyword is the : only keyword (So annoying that I can’t put the : near the o since it makes a :o).  This allows you to only use the parts of the namespace you need.  It wasn’t necessary in this project example, but I threw it in anyhow.  So live with it.

Ok so now everything is ready (Pretty easy huh?).  All that’s left is to prove that I am awesome, and now you have a small bit of said awesome.  Just open a command prompt (windows key; type in cmd; hit enter), and navigate to the project root directory.  Then type lein test.

OH EM GEE FAILURE!!!  That’s right, you just failed to win.  Or won by failing.  Or something actually clever.

As you can see, the reason it failed is that I had the test asserting that the select won’t bring back anything.  However, since I had the insert command, it brought back one user.  And that’s a working SqlKorma example.

Clojure on Windows 7 with Leinigen

If nothing else, this is just a checklist for myself that I’m allowing you, due to my giving nature, to use if you need. This is a very simple bullet point thing. Got help from this post.

Get Java EE:
This is the current link as of this post (11/23/2012)

Create an environment variable:
– Hit the windows key
– Type in envi
– Choose “Edit the system environment variables”
– alt + n (Environment Variables)
– alt + w (New)
– Variable Name: JDK_HOME
– Variable Value: C:\Program Files\Java\jdk1.7.0_09\bin (May be different if you are not running 64 bit windows)
– Tab to “OK” or click it
– alt + s (System variables)
– scroll to “Path”
– append ;%JDK_HOME%;
– Enter (Click OK)
– Enter (Click OK)

Get jar for lein:
This is the current link as of this post (11/23/2012)

Get Wget:
This is the current link as of this post (11/23/2012)

Place both in:
– c:/lib/lein

Create environment variable: (Same procedure as before)
– Variable Name: LEIN_HOME
– Variable Value: c:/lib/lein

Open a command prompt and go to the Lein directory:
– Windows key
– type cmd
– hit enter
– cd c:/lib/lein

Run two lein commands from the command line:
– lein self-install
– lein repl

Those two last commands should get you everything you need to run the Clojure Repl.

Clojure: Default Unit Test Setup

Code for this project is on github.

So you want to test in Clojure, huh? Well good thing you found this oasis of awesomeness. Getting a test up and running is pretty easy, but involves a few simple steps that I will stretch out to the point of pain.

First off, start a new project. Again simple. Just let Linegen to do if for you:


Doing this should create a new project space age tubular frame:


Still with me? Good. Now for the test creation.


As you can see, I added a file with a test. What you might notice, or cause panic, is that I don’t have a file to test with this test file test. And that’s ok, because we’re @#$%ing this $@#% up Test First style. Basically, create a test, and then create what it’s supposed to test. I won’t get into Test First here, but just wanted to calm your concerns.

Now to run the test. This is done by opening up a command window, navigating to the root folder for your project (It will have a project.clj file in it), and typing

	lein test

Ruh roh. Looks like that file that isn’t in existence can’t be tested. Weird, right? Oh well, time to create it.


There are a couple points at this eh point. You will see that I added a file named “simple_math.clj” to the “test_example” folder.

Something I found out while I was creating this little gem of a tutorial; Apparently the convention for a folder is to use a _ where a class in a file uses a -. So as you can see, the folder “test_example” translates the namespace part “test-example”, and “simple_math.clj” becomes “simple-math”. From what I can tell, since I am pretty new to Clojure, lienigen will try to resolve the namespace of “test-directory.simple-math” to “test_directory/simple_math”. I assume this is part of the “convention over configuration” school of thought. Since I come from a .net background where conventions just don’t exist, it caught me off guard.

Anyways, since that is done, it’s time to run the test again.


One failure? Oh yeah, the junk test created by lienegin. Well, just get rid of that:


And run it again:


And things are looking a lot better.