Tag: Python

Mock SQLAlchemy scoped_session query and Why Python is My BFF

sqlAlchemy has sessionmaker to create a session from which you can use query to get whatever you need. For example:

someSession.query(SomeTableRepresentationObject).filter...ect

If you’ve used sqlalchemy, nothing new going on here but it wouldn’t be me if I didn’t point out the obvious and take three sentences to do so.

Now what you may run into is something like this:

def getCountOfUsersByUserName(userName, session):
  return session.query(User).filter(User.userName == userName).count()

Ok now that I typed that out, I see it’s kind of a dumb method. But f–k it, what’s done is done.

Now from a testing situation, this could look tough. After all if you come from a more static language background like I did, you should know how hard things can be to mock at times. (.Net Session… I’m looking at you with judging eyes. Sooooo judging.) But this isn’t a static language, it’s Python.

For purposes of making things less confusing, which is hard for me, when I use the word “Object” I mean method OR class. And remember as always, “banana” is the safe word.

You see, to mock that out all you need is an object that has a query object that takes in something and a filter object on the query object that takes in something and has a count method on it. Yeah that totally makes sense. Try working it backward.

Filter has one parameter and a count method on it. Whatever that parameter is, it doesn’t matter since it is Python. As long as Filter takes in one parameter, you have a winner.

Query, like Filter, takes in one parameter and has a object named Filter on it. Once again, it doesn’t matter what’s passed into Query, just that it takes something in.

Session is basically an object that has no parameters and has a Query object on it.

Now in a static language, this would be annoying. You would need 3 interfaces and mock a whole ton of stuff dynamically with something like Rhino Mocks or just create a bunch of classes of those interfaces that you can pass in. Either way, there are complications. Once you get into things like Web Session or ViewState, it’s a ton of work. Python? Eh, you can do it in 20ish lines. How you ask? Well I’ll show you how!

	class mockFilter(object): #This is the ONE parameter constructor
		def __init__(self):
			self._count = 0
			self._first = dynamicObject()

		def first(self):  #This is the another method that's just coming along for the ride.
			return self._first

		def count(self):  #This is the needed Count method
			return self._count

	class mockQuery(object): #This is the ONE parameter constructor
		def __init__(self):
			self._filter = mockFilter()

		def filter(self, placeHolder): #This is used to mimic the query.filter() call
			return self._filter 

	class mockSession(object):
		def __init__(self):
			self._query = mockQuery()
			self.dirty = []

		def flush(self):
			pass

		def query(self, placeHolder):  #This is used to mimic the session.query call
			return self._query

  #and this... THIS IS SPARTA!!1111... yeah I know, I'm about 3 years too late on that joke.

How does this work? Say I have the method from above:

def getCountOfUsersByUserName(userName, session):
  return session.query(User).filter(User.userName == userName).count()

I could test it using this:

  session = mockSession()
  session.query('').filter('')._count = 0 #Initialize the mock session so it returns 0 from count()

  getCountOfUsersByUserName('sadas', session)

And boom, you have mocking in ten minutes or less or your code is free.

As you can see, the highly dynamic nature of Python makes it a great fit for any project that will need unit tests and mocking. And which project doesn’t? You know what I’m sayin’? You now what I’m saying’? High five!

Note: dynamicObject is …. nothing but a cheesy class that inherits Object but has nothing on it. (Turns out that if I did someObject = Object() I couldn’t do this since Object by default doesn’t contain the ability to add things dynamically… and this was by design.)

And yes, I just quoted myself. I am that awesome.

SqlAlchemy: Self Referential Many To Many

Ok this was just plain annoying to figure out. A couple of misteps and a few F—! later I finally got it. So the situation is this:

You have a table for user to ignore users. So it’s basically a many to many where both sides of the relationship are the user table. The creation might look like this:

class User(Base):
    __tablename__ = "User"
    id = Column(String(36), primary_key=True, default=lambda : str(uuid1()))

Basically a table named User and it has a primary key. Yee haw. Now for the hanging table:

class UserIgnore(Base):
    __tablename__ = "UserIgnore"
    id = Column(String(36), primary_key=True, default=lambda : str(uuid1()))

    ignored_by_id = Column("ignored_by_id", String(36), ForeignKey("User.id"))
    ignored_by = relationship("User", backref="ignored_list",  primaryjoin=(User.id == ignored_by_id))
    ignored_id = Column("ignored_id", String(36), ForeignKey("User.id"))
    ignored = relationship("User", backref="ignored_by_list",  primaryjoin=(User.id == ignored_id))

Basically you need a table/object with two columns: ignored_by_id (The one doing the ignoring) and ignored_id (The one being ignored). And at this point the word “ignored” should look like it’s not even a word anymore because I’ve typed it too much already.

Now the relationship is made using two different properties:

    ...
    ignored_by = relationship("User", backref="ignored_list",  primaryjoin=(User.id == ignored_by_id))
    ...
    ignored = relationship("User", backref="ignored_by_list",  primaryjoin=(User.id == ignored_id))

As you can see, I had to hit it from both angles. (I’d hit it! LOLOLOLHARHARORAALROARH) Not only how to describe the relationship from ignorer to ignoree (Those are words now) but also in reverse. That way I can have a constructor like this:

    def __init__(self, user, userToIgnore):
        self.ignored_by = user
        self.ignored = userToIgnore

And also if I do something like this:

    UserIgnore(_user, _user2)
    UserIgnore(_user2, _user)

I can have it all save correctly just by adding _user to the session or saving _user id at some point.

Also both those lists will be added to the User object without having to do anything further.

someUser.ignored_list

or

someUser.ignored_by_list

Yay for that.

Simple Property Merge for Python Objects

Not sure if this is useful to other people, but then again if I cared that would make me human.

So here’s a quick way to merge two objects in python:

def mergeObjectProperties(objectToMergeFrom, objectToMergeTo):
    """
    Used to copy properties from one object to another if there isn't a naming conflict;
    """
    for property in objectToMergeFrom.__dict__:
        #Check to make sure it can't be called... ie a method.
        #Also make sure the objectobjectToMergeTo doesn't have a property of the same name.
        if not callable(objectToMergeFrom.__dict__[property]) and not hasattr(objectToMergeTo, property):
            setattr(objectToMergeTo, property, getattr(objectToMergeFrom, property))

    return objectToMergeTo

This is a good example of how the whole dynamic addition of properties happens in Python. The objects in this example have a __dict__ property that is basically where the property/method/other stuff information is held in dictionary form. What does this mean? It means that if you use the setattr method you can then later call the property like you would normally.

  setattr(someObject, 'someProperty', 'some value')

  someObject.someProperty == 'some value

Knowing this is important as it allows you to really do some fun dynamic stuff that a lot of other languages just don’t allow.

I Have Found Python and I Am a Changed Man

Ok so maybe the title is full of sensationalism, but in some ways it’s true.

In the beginning there was Microsoft.

When I started programming at the unusually late age of 24, I was brought into the world by Microsoft. At the time, ASP was still in fair use and .Net was the new wonder drug and no one, not even Microsoft, could nail down what it was exactly. It was a miracle in a bottle, an olde time elixir. It would fix everything known and even unknown unknowns. Now personally, I had only really taken a liking to the style of C++. I preferred brackets to subs and never minded semi-colons. So you can imagine when I found out about “the C#” , I was all about it. Looking back now, it does seem sort of silly since there really wasn’t that much of a difference between VB.Net and C#, but d—-it I was going to get my semi-colons or here go hell come. Naturally this meant I would start a journey of 8+ or so years into the Microsoft frontier.

Have Carrot and Stick, Will Travel

I think one of the strongest suits of the .Net framework was Microsoft’s need to constantly improve it. Now these improvements weren’t overnight but it seems like they made a lot of improvements in relatively short time. Every version just seemed to have the answer to every question I didn’t know I needed to ask. It was like they were reading the mind of future me and adding features based on that mind. Yeah ok, it’s a little ego centric to suppose that future me was the only person asking for such things, but I’m cool with my version. I can still remember how annoyed I was that the place I was working for wasn’t going to invest in 2.0 for a while, thus keeping me from my coveted generics. (Pretty sure I said the word to the point it didn’t sound a like word anymore) And honestly, a little nerd rage was expected since I didn’t want to type every parameter as Object. I can also remember how in awe I was of Linq and some of the semi-functional programming concepts being introduced with 3.0/3.5. It seemed like as soon as I started to covet something new, BAM it was there. So really, I had no reason to ever stray from .Net because I was always well taken care of. Course then something changed and it was Microsoft’s fault.

There some things you can’t unsee

Remember that thing named Linq? Well Linq was more in line with the whole .Net mystique. It wasn’t just a way to manipulate lists, it was a whole lot more. With Linq came a few nice concepts like typed delegate, anonymous types, type inference on a much larger scale, extension methods, and a whole slew of other larger concepts like Entity Framework. It was kind of like taking a trip to the moon. Sure the whole moon thing is great, but the massive amount of new technologies that came from just getting there was where the real gold was. In the development of Linq, Microsoft had to come up with new stuff to even run the concept of Linq. At some level it was touted as Microsoft’s attempt to bring a more dynamic and functional look to C#. Now no one was claiming that C# was a functional language but d—ed if it didn’t try. After all, it’s not easy to make something more dynamic when the underlying technology is meant to be more static. Between being able to easily pass methods around like objects and using lambda expressions to simplify dealing with lists, it was a huge step forward and personally a large shift in my programming paradigm. (Yes I just used paradigm in my blog.) So where’s the catch? It’s simple, my eyes were opened to stunning new concepts. The use of Linq, lambda expressions, and Func/Action got me to start looking at programming as a whole in a different way. I wanted to be more dynamic. I wanted to start solving old issues using all this exciting stuff. And because of this I started expecting more and pushing .Net to the limits. For the first time in my programming career, the carrot was chasing me.

Model View what?

If Linq was the straw that broke the camel’s back, then MVC was the bus that ran it over by mistake. For years I had been in the standard world of Web Forms. Something that at one time seemed to be a beautiful bridge between the web and stand alone projects was now looking worse and worse. All the issues I had with Web Forms were now being magnified by the overall streamlined approach with MVC. It was such a strong double-hit between Linq and MVC that I just couldn’t ignore the outside world anymore. It’s sort of like that moment when you realize that your parents are human just like you. It makes you think that maybe everything you knew wasn’t actually the only way to do things. That maybe there’s something more out there. Maybe, just maybe, people didn’t eat cold pizza with syrup. This was the moment when I started wondering if there was something greater out there. (Just like Vger) Because I had never dabbled in anything outside of .Net (except a regrettable affair with PHP that still worries me that someday a little PHP will show up at my door claiming that I’m it’s dad) so this concept was brand new to me. Of course anyone who wasn’t completely clueless like me would have known that MVC had been used by things like Java for quite a few years. To makes matters ever worse, the poor camel was struck by a huge meteor named Javascript. Yes, I typed Javascript. With MVC came my heavy use of Javascript and it’s much fancier suit and tie JQuery, and guess what? I noticed that most of the cool new dynamic stuff for .Net was being used in Javascript all the while. I had shied away from Javascript for a long time since I was brainwashed into thinking it was evil. Just use post backs and server side code. Stay away from the language that we dare not speak its name. Problem was, it is almost a need when using MVC unless you want to refresh the page on any form post. Long story short, I was reeling from all this new information.

Every silver lining has a dark cloud

One thing that I didn’t like about most examples of how the model and view work together was the use of a dictionary to hold values that would appear on the view. I’ve never liked magic string programming so I naturally went with typed views. This meant that for every view I would have a model class working alongside to help pass values in a much more clean and safe way. The issue? A whole f–king lot of models. So much that it almost became a separate project in itself to hold all of them. Something just didn’t seem right about this. One of the nice things about C# is it’s more rigid feel to keep people from screwing things up (unlike the magic string theory). Problem was, for rapid development this just didn’t cut it. And let’s be honest, in the real world (unless you are the lucky few that have decent deadlines, i.e., top-end jobs) rapid is all we get. This left me in an uncomfortable place. We’re talking showing-up-to-school-naked-and-realizing-it’s-not-a-dream-this-time uncomfortable.

In time all things become clear

It would seem natural that if I liked all the more dynamic features of .Net and Javascript, I would possibly try to find a language that suited my needs. Well there’s an old thought in psychology: People don’t change unless something makes them. And as much as I’d like to say I braved the new world on my own, it really was more of a wake up call. While Microsoft has done a lot to add needed complexity to the framework, they continue to add far too much simplicity.

Not to long ago, some time after 4.0 was just about to go “gold” I read a couple articles on how easy programming was becoming in the .Net world, and how salaries for .Net were starting to drop. To sum it up, Microsoft has always tried to walk two lines: Cater to the more expert programmer AND the drag-and-drop programmer. Some nice examples would be from 2.0 in generics and update panels. Generics are still not understood by drag-and-drop programmers (as I have been shown on many occasions) but man they love their update panels. I mean it’s ajax without actually knowing anything about it. And I think this is where .Net is going to head. I think as elitist programmers we’d like to think that knowing design patterns and how to successfully design a streamlined system is important but honestly, it isn’t. And I’m saying this as a person who has had 7+ jobs jobs in the last 9+ years. Businesses want results. They don’t care if it costs them more in the end if you hack something together. I’ve never really met a company that thought that far ahead, and in some ways I can understand this. Getting to the market first can be a huge advantage. The “Just get something out now and worry about it later” mantra is prevalent in all walks of business. It’s hard to convince companies to take more time to develop a more solid system. Because of this, things like update panels and drag-and-drop design are still hanging strong which by proxy means having a solid understanding of how things work really doesn’t matter as much.

Though way too lengthy of a side bar, the point is that this is the event I needed to change. Because Python is a more fringe language but is used by some really good companies, it seemed like a reasonable switch… or at least something to experiment with. There is nothing really drag-and-drop about it.

If at first you don’t succeed…

I will admit that at first I was a little apprehensive about Python. It just didn’t seem to offer anything that C# didn’t… again at first. However, the more I used Python and Pylons, the more free I felt. All those years of pushing and expecting rigidity were washing away. Don’t me wrong, it took a while to really get how powerful Python can be. I mean stuff like adding properties to objects dynamically was nice, but that was also in 4.0. Passing methods like objects: C#… well at least with some restrictions. But with that being said, having options like that without the need for typing parameters and done so with a natural feel made it pretty well suited for web development. I’m not going to spout the virtues of Python and I’m pretty sure I’ve derailed this post quite a bit, but the moral of the story is simple:

There’s more out there than Microsoft/.Net. Now I realize that probably 2/5 readers (not a fraction but an actual guess on how many people will read this) will be thinking ‘DUH’ but this isn’t really written for people who know this already. This is written so that even Microsoft slappies like me can break free of a what I thought was the be-all and end-all, and really learn something new that isn’t new to .Net. I’ve built my career around .Net and have no doubt it will most likely be what I’m paid to use for a while. With that being said, there’s absolutely nothing wrong with trying new paths. Is Python the language to do this for you? I have no idea. Maybe it’s Ruby or (maybe(the(language(is(Lisp))))). That I can’t tell you. Only thing I can tell you is that you won’t know until you get out and try something new. You can’t possibly know what’s out there if you stay at the house that Microsoft built. In a world with too few eggs and a ton of baskets, you really can’t afford keep things the same.

Python: Added Folder By Namespace Won’t Import…. How To Make A Namespace.

Ran into this last night and it was giving me some issues. Basically, I added a folder (unitTest) and then some .py files to it. So my folder structure is something like:

pyzzazz/unitTest/validation/validationTest.py

Now one would assume you could do this:

from python.unitTest.validation.validationTest import someTestMethod

Problem was that the IDE (PYCHARM PLUG) was telling that it couldn’t see the unitTest namespace. After some nerd rage and searching, turns out that Python by nature doesn’t assume .py files outside of the main folder structure “belong” to it. The fix is simple though, add a blank __init__.py file to the folder. In my case, I added a blank __init__.py file to the unitTestFolder. Then and only then would the above namespace actually work. Here’s something on the __init__.py file

Python, Pyramid, and Mako… How to get Mako Working With Pyramid…

Well looks like Pylons is now Pyramids, and here go hell come. Now I could go on explaining the differences (Which most likely I will once I get farther in), but this is more just a quick post.

ANSWER:

In the development.ini file, place this under the [app:pytrends] area

mako.directories=projectName:folderWhereYourTemplatesAre

For example, the project I have is pyzazz (Don’t ask me what that is cause I don’t even know) and the template folder is… well templates. So I have it as:

mako.directories=pyzazz:templates

MORE USELESS WORDS:

When using pylons I used mako. It was fast and well it was by default. Now the default template engine is Chameleon which (Sorry Chameleon creator) sucks. First time using it, it was throwing parser errors in javascript lines like:

 if (urlParams.actual && urlParams.actual == 'true')

and it forced me to end an input with the </input> tag. And if you know anything about HTML, input tags DO NOT HAVE END TAGS. True story, go check w3.org if you feel like. Little things like that just annoy the hell out of me. Yeah I know it’s just html and javascript, the runts in the litter as far as programmers care, but you still can’t jack things up like that. And on that note: if someone who uses chameleon (Or helped write it) actually comes here, you are welcome to shoot down my claims.

Python… DOUBLE TEE EFF Exclamation mark one one

Fair Warning: This is all conjecture and idiocy… possible more the latter than the former. With that in mind, this is more of a flow of thought than any kind of organized scientific paper… thing.

NOTE: When I use the word “attribute” I really mean things like properties, fields, methods, ect on a class. Since I am guessing most people reading this come from a C# or Java background, the word “attribute” might be confused with something else. So remember attribute = property/method/field/anything that defines an object.

So as I delve farther and farther into the insanity that is Python, I can feel my mind screaming. Not like “Oh f— I’m being eaten alive” kind of screaming, more like “Don’t open that door! You know the guy with the axe is behind it!” Of course, like any good horror movie character I’m too dumb to know better. So delve I do.

Now I’m what you could consider a classically trained programmer and by that I mean I never programmed before college so I didn’t do much exploration. You know, a Microsoft programmer. With that, I’ve had the ideas of interfaces and class hierarchies complete hammered into my head, so it shouldn’t be any surprise that I’ve tried my best to make Python into c#. After all, that’s what I should be doing. I don’t know any better. DONT JUDGE ME!
It wasn’t until the other day when it just hit me. And I don’t mean slight hit, I mean like I just told Miguel Cabrera I’ve seen better players in Little League. It occurred to me that it’s possible, just possible I might have to change my way of thinking because of one major design difference: dynamic attributes… actually f— it, dynamic everything, but let’s just stick with something simple. In Python something like this:

  someObject = dynamicObject()
  someObject.propertyImJustAddingForTheHellOfIt = 1

Where dynamicObject is nothing but a cheesy class that inherits Object but has nothing on it. (Turns out that if I did someObject = Object() I couldn’t do this since Object by default doesn’t contain the ability to add things dynamically… and this was by design.) So what does this mean? Well it means that anything at anytime could have any attribute. You might be asking why this is such a big deal. Annnnnnd that’s why I’m typing this out so stop asking questions until the end, k?

If any object can have anything on it at anytime, it could be said that types and classes kind of go out the window. Why? Because when it all gets washed away, what really does the work in a program? Methods. Now with C# there is a much stronger enforcement of typing so a method KNOWS that whatever coming in has to be a certain type. This is where object inheritance and interface implementation come in. Because there is such a prerequisite for typing in C#, you have to develop around the object itself and have the methods conform to them. In Python, I question whether this approach is really warranted since with dynamic attributes what really matters is what the method needs. It could be argued in the world of dynamic, the methods actually dictate everything. Since types are basically thrown out the window, what’s the point? Why not just shift all development around methods and their results? It’s kind of an inverse way of looking at things when used to the C# way of doing them.

Method Oriented Programming?

This is something I’m kicking around a bit, but why not base the architecture around methods rather than classes? It would give a much truer sense to the word “Factory”. Instead of worrying about having a class hierarchy, have methods responsible for creating objects on the fly, piecing them together with other methods and such. I would think this could give a program a massive ability to adapt quickly to any given situation if you can add and subtract from objects on the fly since you are not longer concerned with types, just attributes. Does the thing coming in have X attribute? Yes? Great do something. No? Fine don’t do something. (Python has a built in way to check if an object has a certain property) Does it need it to continue? Well call a method to add it to the object. Has the use for it expired? Well just remove it. Taken to an extreme, I could see this being used to construct more self sustainable programs, ones that can makes choices on their own to produce novel outcomes. (Sadly, used like viruses and hacking security systems come to mind, but there has to be more noble uses for this) Sounds like a lot of freedom.

Now I’m not saying you can completely get away from class structure… at least I haven’t gone far enough into this concept to see if it’s possible. However, it should be obvious that it at least allows for a completely different way of thinking that may help solve problems that were seemingly impossible to solve using more rigid languages like C#. Of course, it also lends itself to shooting yourself in the foot. Most likely, this approach would need more overall care since you don’t have things like type safety to cover your a–.
Anyways, this is just preliminary rambling of a child (programmatically speaking… and maturity wise) who is just kicking around ideas that weren’t really there before.

An odd side note, python gives a new… or maybe the correct… meaning to the word “constructor” since because of that highly dynamic nature it’s not unlikely to see something like this:

  __int__(self):
    self.someNewField = "This property now exists"
    self.someNewMethod = someFile.someMethod

Which is actually more like a factory then how say c# uses constructors, which more the most part is like a initialization. The other interesting thing to note is how python treats files and methods. If you declare a method on a file, you can actually import that method as if it were an object itself. This makes the above code even easier to accomplish since you can have a file of just methods that you can “import” and add to an object.