Wednesday, March 7, 2012

Python from scratch- RegEx

It was 5 weeks ago when I first published my first post about me learning Python from Scratch and since then also published Part 2 and Part 3 posts.
To study Python On-Line I am using a Google's Python class site and I follow it step by step.
Each class takes me to read and understand around 2 hours and to do the practice code around 4-5 hours more; The thing is that I do not use these 7 hours in a row but usually over one week and this is not because I am lazy but because I do it during the night instead of sleeping and when my boys are actually sleeping and not ill.

Last night I learned Regular Expressions (= RegEx) and boy! It was interesting to see how you manage to pull a text out of a large text file.
For those of you who don't know me (or haven't read my profile yet) I am working in a security company as a QA manager. Our product needs to detect text file leaked out of the perimeter of any organization. The code for the detection is using RegEx over Python and that is a good reason for me to improve my RegEx skills and of course Python.

The class itself is confusing, I mean- you know what RegEx is and you certainly understand why someone can use it, but it is damn hard to remember the patterns (and so far they are only the basics) and I also found some illogical parameters here:
-           For some reason \w is a word char and \s is a whitespace while in my mind \s should be string and \w should stands for whitespace. Bizarre.
-           I would expect Repetition to act like Wildcards but ' + ' is not a wildcard.

Yet, this tool is very powerful and I am envy in people controlling it just like our research guys who write these RegEx detection codes.

When I felt ready I came to solve the exercise (linked here) which is long to read but once you see an example of the needed output it becomes easier to understand.

So here is my code, I am proud of it and although it looks short it was not short at all in time and I had some tough moments here:


I have attached also Google solution which is different then mine; it is ok to be different but what I did take from their solution is how to handle a case of 'invalid data’ or ‘unexpected data’ unlike my assumption that the data is valid and fits:



Apr/17/2012- Next post here.

8 comments:

  1. You ain't seen nothing yet.

    ReplyDelete
  2. awesome! keep up the good work!

    ReplyDelete
  3. I just started MIT's OCW "Introduction to Computer Science and Programming" which is a Python CS course. I had no idea Google had a Python class. Thanks!

    Just in case you want another reference for your Python education:
    http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-00-introduction-to-computer-science-and-programming-fall-2008/index.htm

    ReplyDelete
    Replies
    1. Now i will surely need it.
      Thanks

      Delete
  4. I'd recommend PyParsing (http://pyparsing.wikispaces.com/) to anyone interested in more readable and easier to support regular expressions in Python.

    ReplyDelete
  5. A site which i like to point out to new pythonics is -> http://www.pyschools.com
    Keep on going and good luck :-)

    ReplyDelete
    Replies
    1. Thanks and keep following me :-)

      Delete