Library conflict between Datejs and D3

I’ve had fun over the past few days tracking down a problem where D3 Transitions weren’t working correctly. Everything looked right and I was pulling my hair out trying to figure out why the transition didn’t get invoked. Copying the code in question to a separate page (in isolation) showed that the transitions worked fine, so I figured it must be a conflict with something else on the page.

After a couple hours of deleting things from the page (it’s tough to pull things off because of the tree of dependencies) I figured out the problem was Datejs. A little googling confirmed it. What made this challenging was that there wasn’t any errors from the conflict. It just didn’t work.

I’m not clear on what the cause of the problem is (I had already lost enough time), but I ended up switching everything to moment.js. Datejs looks like it’s been dead since 2008 anyway.

Connecting to the Marketo SOAP API using Python and suds

Marketo has a SOAP-based API that allows you to interact with a lot of their data, and while there is a Python library that supports it, the library doesn’t cover nearly all the methods supported in the API.

suds is a Python SOAP library that will read WSDL and provide methods for calling into the service.

Here is an example of making a call to the getLead(…):

import hmac
import hashlib
import datetime
import time
from suds.client import Client

def _utc_offset(date, use_system_timezone):
    if isinstance(date, datetime.datetime) and date.tzinfo is not None:
        return _timedelta_to_seconds(date.dst() or date.utcoffset())
    elif use_system_timezone:
        if date.year < 1970:
            # We use 1972 because 1970 doesn't have a leap day (feb 29)
            t = time.mktime(date.replace(year=1972).timetuple())
        else:
            t = time.mktime(date.timetuple())
        if time.localtime(t).tm_isdst: # pragma: no cover
            return -time.altzone
        else:
            return -time.timezone
    else:
        return 0

def rfc3339(date, utc=False, use_system_timezone=True):
    # Try to convert timestamp to datetime
    try:
        if use_system_timezone:
            date = datetime.datetime.fromtimestamp(date)
        else:
            date = datetime.datetime.utcfromtimestamp(date)
    except TypeError:
        pass

    if not isinstance(date, datetime.date):
        raise TypeError('Expected timestamp or date object. Got %r.' %
                        type(date))

    if not isinstance(date, datetime.datetime):
        date = datetime.datetime(*date.timetuple()[:3])
    utc_offset = _utc_offset(date, use_system_timezone)
    if utc:
        return _string(date + datetime.timedelta(seconds=utc_offset), 'Z')
    else:
        return _string(date, _timezone(utc_offset))

def _string(d, timezone):
    return ('%04d-%02d-%02dT%02d:%02d:%02d%s' %
            (d.year, d.month, d.day, d.hour, d.minute, d.second, timezone))

def sign(message, encryption_key):
    digest = hmac.new(encryption_key, message, hashlib.sha1)
    return digest.hexdigest().lower()

def set_header(client, user_id, encryption_key):
    h = client.factory.create('AuthenticationHeaderInfo')
    h.mktowsUserId = user_id
    h.requestTimestamp = rfc3339(datetime.datetime.now())
    h.requestSignature = sign(h.requestTimestamp + user_id, encryption_key)
    client.set_options(soapheaders=h)

url = 'pointer to your api service here?WSDL'
client = Client(url)

set_header(client, 'your username here', 'your secret key here')

leadKey = client.factory.create('LeadKey')
leadKeyRef = client.factory.create('LeadKeyRef')

leadKey.keyType = leadKeyRef.EMAIL
leadKey.keyValue = 'bob@dole.com'

print client.service.getLead(leadKey=leadKey)

Note that some of the above code is copied from/based on the aforementioned marketo-python library

pdf2htmlEX

pdf2htmlEX is a great library for converting PDFs to HTML5 pages that can be viewed directly in browsers and on mobile. Text from the document is extracted as selectable text in the rendered document. It’s a C++ codebase built on top of the Poppler PDF library. Check out an example generated PDF here.

MuPDF

MuPDF is a great command line PDF processing utility built on Ghostscript. It provides the ability to extract text, images, and fonts via its pdfextract utility and the ability to render pages of the PDF as images via its pdfdraw utility.

Install it on Mac OS X (via Homebrew):

brew install mupdf

Also available on Linux via various packaging systems and Windows via a separate installer.

PDFBox for Reading PDFs in Java

Apache has a cool library PDFBox that lets you work with PDF documents in pure Java. It allows you to generate PDF documents, read them, extract text, and generate images from the documents. I’ve been using it for thumbnail generation and it has worked well, though I haven’t done performance tests comparing it to other libraries out there.

CrocoDocs to Offer HTML5 PDF/Doc Converstion

CrocoDocs announced that they are offering HTML5 document conversion for PDFs and Word documents. Interesting because it is relevant for my startup Docalytics.

Setup and Teardown for QUnit Tests

The QUnit documentation talks about how to define setup and teardown methods, but fails to give a code example. Here is a quick one for reference:

module("Module Name", {
	setup: function() {
		// setup logic here
	},
	teardown: function() {
		// teardown logic here
	}
});

test("Some Test", function () {
	ok(true, "test code here");
});

Setup and teardown functions are defined on a per-module basis, via the “lifecycle” object. This object just has methods “setup” and “teardown” as shown above. Pretty simple.

OCaml to JavaScript

I’ve always had a soft spot for OCaml, today I found an OCaml to JavaScript compiler. This could be interesting to use with the Opa framework.

BugAid

I’m excited; I just read this blog post about BugAid, a Visual Studio add-on that has many features I’ve often wanted myself. Chief among them is the ability to search through a live object for data. When you are working with an unfamiliar code base and you are trying to track how data flows through the system, this can be very useful. Another cool feature is that it shows the partial evaluation of expressions within an if statement. Very cool. Going to install the beta today.

Taking a Screenshot From Scala

This is a bit of a cheat, it’s just really using the Java libraries from here and here.

import java.awt._
import java.io._
import javax.imageio._

def captureScreenshot(filepath : String) = {
  val screenRect = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize())
  val capture = new Robot().createScreenCapture(screenRect)
  ImageIO.write(capture, "png", new File(filepath, "screen.png"))
}