ListFilter Prototype

Multiple space terminated regular expressions for list filtering.

The examples presented are somewhat hare-brained. a) Cause I'm hare-brained b) I'm really tired at the moment. Note the dramatic pauses :) Can.....you.....see....what....I'm.........do....ing You can imagine the usefulness though.

The filter is a prototype implementation of a filtering syntax for a QuickOpen Panel specifically designed for `quick` (try not to laugh) *multiple item* selection. The idea is to allow you to just keep typing in to refine your search. NOT this OR that. NOT that. `Open All In New Window` etc.

It is implemented using the editors actual text buffer API.

My editors current QuickOpen panel (while it does support multiple selection using ctrl-enter) works a lot like the FireFox url history search; space terminated tokens that each list item must match else will be filtered from the list. This works great for selecting one item but what if you want to open up multiple items at a time?

Plain regex searches are too unwieldy and lack the speed of entry desired.

The filtering is a regex extension of the current `all space terminated word chunks in any order`. It maintains most of the benefits and generally works as before for non regex characters. If you just need to type in some alphanumerics then it should be just as quick. It's essentially multiple regexes instead of multiple substring matches.

OPERATORS:

' ' AND operator
! NOT operator
| OR operator

jQuery = $;

I have been making a foray into JavaScript recently for work and having had good experiences with `jQuery Lightbox` decided to use it for the basis of a job I was doing.

It is essentially a carpet gallery website with collections of colors (1:M). The designer wanted it to be `AJAX` ( a term that seems to have been hijacked to mean any page updated without slow browser refresh )

In the middle of the page is a large image. Above it are next/prev collection links and tiny `swatches` (thumbnails) containing links to each color for the current selection.

To either side of the image are next/prev color in collection links and upon mouseover a window will appear showing a magnified area following the cursor which is changed to a crosshair. The cursor will change via css styling to `cursor:wait` whilst waiting for the zoom image to load.

Upon changing color (via swatch, next/prev collection/color ) an animated gif will show while the chosen color's image is loading.

At first I was keeping a counter of the current colors position in the collections array of colors (clicking on the little thumbnails would take the title attribute, slug it and use that for the color, updating the current position index by with an $.inArray(color, colors) )

The filenames to load was, and are, a function of current collection and color.

The problem was having the state in an internal counter didn't really work for `open in new tab` or for sending links. "Hey check out this carpet... no the red one... Did you type in that url properly? Just paste it in."

I did some googling for `ajax urls` and stumbled upon a technique that sounded useful. That being polling the hash location for changes and then setting page state as a function of the hash. They said the ideal `polling` rate was 100ms. Sounded pretty hacky but at least the urls worked.

I searched for and found a plugin for jQuery that allows you to set event handlers for when the window hash changes. It uses polling but is responsive (42 ms) and works on IE 6.

I therefore just set a callback to update all the links and the image upon hash changing. I split the hash on '--' to find the current collection and color

The great thing about it is that the event is fired on page load so it goes through the `change color` routine. Updates all the links, shows the loading animated gif etc.

You can send links to people, open in next tab from any of the links etc

jQuery and its plugins made everything really straight forward. The only head scratching was getting the magnifying glass to work. None of the plugins worked out of the box for images that changed src. They worked mainly for a `static` gallery.

CTags 2: TDD

This is part two of CTags. See part 1

Test Driven Development

Adherents religiously write tests first for *every* function they write.

Personally, I think a tonne of unit tests while prototyping is a waste of time for `simple` stuff with only one programmer working. It tends to feel like walking in mud. I prefer higher level tests that while they may not pinpoint the exact cause of failure won't double refactoring efforts. Especially when prototyping. Spend ages testing that `The Wrong Way` works? No thanks... Prototype, then rewrite with tests.

Sometimes though, `TDD` really is indispensable while exploring. Especially when working on stuff that is pushing the limits of your understanding. If I start encountering bugs I usually see it as a sign I need to start writing tests. Rather than using transient print statements to debug I'll write some unit tests.

The TagFile class below (now commented) is an example of when I found testing while prototyping invaluable.

  1  #################################### IMPORTS ###################################
  2  
  3  from __future__ import with_statement
  4  
  5  import os
  6  import bisect
  7  import mmap
  8  import unittest
  9  
 10  ################################### CONSTANTS ##################################
 11  
 12  
 13  """
 14  The tags in a `tags` file are listed one per line formatted as so:
 15  
 16      tag_name<TAB>file_name<TAB>ex_cmd;"<TAB>extension_fields
 17  
 18  """
 19  
 20  # symbolic constants for column indexes
 21  SYMBOL = 0
 22  FILENAME = 1
 23  
 24  ################################################################################
 25  
 26  class TagFile(object):
 27      def __init__(self, p, column):
 28          """ 
 29          
 30          Instantiate a new TagFile
 31  
 32          @p            path to `tags` file
 33          @column       which column to read
 34  
 35          """
 36  
 37          self.p = p
 38          self.column = column
 39  
 40      def __getitem__(self, index):
 41          "self.fh is the mmap opened by get"
 42          # Seek to a certain byte index
 43          self.fh.seek(index)
 44  
 45          # The position is likely to be halfway through a line so read up to
 46          # the first new line and throw away the `junk`
 47          self.fh.readline()
 48  
 49          # Note that it's actually returning the column from the line *after* the
 50          # line region containing the index.
 51          
 52          return self.fh.readline().split('\t')[self.column]
 53  
 54      def __len__(self):
 55          # bisect.bisect_left search must know how large the file is
 56          return os.stat(self.p).st_size
 57  
 58      def get(self, *tags):
 59          """
 60          
 61          Get all lines for one or more tags
 62          
 63          """
 64  
 65          with open(self.p, 'r+') as fh:
 66              # mmap is not needed but delivers performance increase
 67              self.fh = mmap.mmap(fh.fileno(), 0)
 68  
 69              for tag in tags:
 70                  # As __getitem__ returns the colum from the line region *after*
 71                  # that containing the index pt then bisect( alias of
 72                  # bisect_right ) will give the wrong index.
 73  
 74                  b4 = bisect.bisect_left(self, tag)
 75  
 76                  # Move the file to the position found
 77                  fh.seek(b4)
 78  
 79                  # Iterate over file. There may be more than one tag line to get
 80                  # per symbol/filename
 81                  for l in fh:
 82                      # Compare search vs line at column
 83                      comp = cmp(l.split('\t')[self.column], tag)
 84  
 85                      # This handles the case of being `left of left` due to 
 86                      # __getitem__ index being left of symbol it returns
 87                      # ie wait until catch up
 88                      if    comp == -1:  continue
 89                      # If line is greater then have moved on to next symbol
 90                      elif  comp:  break
 91  
 92                      # Found tag!
 93                      yield l
 94  
 95              # close mmap
 96              self.fh.close()
 97  
 98  ##################################### TESTS ####################################
 99  
100  class CTagsTest(unittest.TestCase):
101      def test_tags_files(self):
102          """
103  
104          This test basically iterates over each line in the tags file creating
105          a list of lines for each unique symbol it finds. It then compares this
106          list to that returned by the TagFile binary search. 
107  
108          """
109  
110          # Successfully passed test on 10MB+ tags file
111          # tags = r"C:\Python25\Lib\tags"
112          
113          tags = 'tags'
114          tag_file = TagFile(tags, SYMBOL)
115  
116          with open(tags, 'r') as fh:
117              latest = ''
118              lines  = []
119  
120              for l in fh:
121                  symbol = l.split('\t')[SYMBOL]
122  
123                  if symbol != latest:
124  
125                      if latest:
126                          tags = list(tag_file.get(latest))
127                          self.assertEqual(lines, tags)
128  
129                          lines = []
130  
131                      latest = symbol
132  
133                  lines += [l]
134  
135  if __name__ == '__main__':
136      unittest.main()
137  
138  ################################################################################