TRANSCODE

Trans Technology and Programming Blog

Lisp Rainbows

I always thought Skittles and Lisp should have more in common. Perhaps it's just me. Perhaps I am the only one that has trouble following all those damn parentheses. It doesn't take many of them, honestly. Three of four levels in and I start to get a little dizzy. Here's an example I was just looking at this morning, from Racket's Quick Introduction.

(define (checker p1 p2)
  (let ([p12 (hc-append p1 p2)]
        [p21 (hc-append p2 p1)])
    (vc-append p12 p21)))

Now, assuming that others do in fact have the same difficulty (and love for Skittles) has it not occurred any of the creators of syntax highlighting for Lisp, that perhaps it would be a good idea to color coordinate the parentheticals?

Lisp the Rainbow

Oh, yea. You can almost taste that Rainbow!

Written By
trans —

The Leaning Tower Of Markdown

You would think that any reasonable human being endeavoring to invent a light-weight markup format would look at _foo_ and naturally conclude without a second thought, that if it were to do anything at all it would render as an underlined foo. But apparently, if you happen to be John Gruber and the creator of the most popular markup format in use today, Markdown, that cold case of obviousness is somehow completely wrong. Gruber instead decided it should do the exact same italic emphases as *foo*.

Sad little _foo_. How sad for you.
To be underlined is your sole pursue.
But regulated to lean, like an Italian Pisa
it would seem, never to truly be
a truly proper underling. Oh, the irony.

By the way, Redcarpet, Github's markdown parser, does support an :underline configuration option. It sure would be nice if they turned it on! And if others followed suite.

Written By
trans —

Yaml Logic

Expressing Logical Relations in YAML

When designing YES, YAML Easy Schema, it became clear that the specification would have to support logical relationships. Now, that seems a rather tricky thing to try to express in YAML. And my first thought was that I'd have to resort to some level of string parsing, which did not sit well with me. I wanted the whole spec to be YAML all the way down.

Eventually I realized that it was in fact possible. The trick was to use special logical types: !and and !or. Then it is easy enough to create logical graphs.

example: !and
  - this
  - that
  - !or
    - these
    - those

In addition we can add a !not type.

example: !and
  - this
  - !not that

And that about wraps it up.

Has anyone else come across a need for this sort of thing? Any ideas for improvements? Any caveats I overlooked?

Written By
trans —

Drink Me

My Critique of Elixir

First, let me admonish myself for having limited knowledge of the subject. I am far from an Elixir expert. So I readily defer to those with more experience that can make a solid contrary argument against my conclusions. However, as I am unlikely to continue on in my adventures with Elixir, this is clearly the only time I have left to opine on the topic. So I will do so, and feel free to take my opinions with as much salt as you feel necessary.

There are many things to like about Elixir. Most notable of them is it's inheritance from Erlang and it's improved syntax inspired by Ruby. Its Erlang roots bring so many great features to the table: true functional programming, rock solid stability, distributed computing, and so on. Anything that is good about Erlang, is likewise good about Elixir. On top of this, Elixir brings a much more pleasurable syntax and a few higher-level conveniences which really set it apart from Erlang. (Otherwise there would be no point but to use Erlang directly). But then there are the caveats.

On the minor end, Elixir's code documentation is a bit odd. Documentation of modules is placed within the defmodule block, while documentation of functions are placed on the outside the def blocks. Why the difference? In addition there is no sigil at the beginning of documentation lines to clearly indicate a comment. If a documentation section were large enough, one could mistake examples for actual program code. Only by the magic of syntax highlighters are we saved from any such "at a glance" error.

Then there is the minor issue of the def prefix on the definition of everything. defmodule, defstruct, deftype, etc. One can understand the desire to avoid too many keywords, but surely module, struct and type would do much better, as they do in almost every other language, and can be avoided as keywords if need be by appropriate context.

A little more troubling is the awful block syntax. Despite all the inspiration form Ruby, Elixir did not adopt Ruby's elegant block syntax. Instead we get a strange twist of fn keyword, an arrow -> and an end keyword, all within the argument parenthetical.

  Enum.each(list, fn x ->
    ...
  end)

It feels a bit more like Javascript than Ruby. Ironically, this ugliness doesn't save us from a whole mess of seemingly extraneous do statements everywhere. Every def* and even every condition like if also requires a do to define the subsequent block.

  if ok?() do
     ...
  end

It might not seem like much, but when you write enough code and find yourself having to always fix missing do's first off, you can't help but feel as if your time could be spent on more useful endeavors.

On a higher level of critique, I think Elixir, like Erlang and their ML brethren, suffer from Overtype Syndrome, if you will allow me to coin a phrase. Because the language is both functional and so strongly typed, every method comes by way of a module. So instead of say, list.each{...} as in Ruby, one much specify where each is defined, Enum.each(list, ...). And because one must repeat these module names over and over, there is a naturally common practice of extreme abbreviation. In some fashion it's nice that names are concise. Yet it can be a bit difficult encountering module names like IEx in passing. In addition, we end up with a whole slew of modules and types with which to contend. Of course this can be true of any strongly typed language, but it seems especially so for the ML branch of languages.

Finally, my last niggle is with functional programming in general. Honestly, it's pretty awesome. But just as honestly, it is difficult. The lack of procedural techniques puts a significant onus on the functional language to provide a large set of easy to understand and easy to utilize functions to pipe together to achieve the desired result. Partly it is due to lack of experience in thinking functionally, but it is clear enough that even for very component functional programmers it can be a real challenge to piece together the right functions to solve a complex problem. I believe this is because functional programming forces you to think about the whole more so than procedural programming. With a procedure, one can quickly break down a problem into parts. e.g. "I have an array of strings, which I need to iterate over and extract each word, and for each word, make it lowercase, and then update a hash table to track the number of times the word occurs." Easy enough. Functionally the reads more like, "I need to build a dict via a flat-map over a list of strings and the list of words in each while lowering the case of each word, with an updated key/value pair." Or something like that. It is simply harder to even to talk in functional terms.

Written By
trans —

Next Level Please

I wish my computer was more integrated. There are a number of things I want it to do right "out-of-the-box" and I want them all to come to me through a single integrated interface.

  • Authenticate id
  • Manage privacy
  • Tell time and keep a calendar
    • Set reminders and alarms
  • Provide weather report(s)
  • Map w/ Global positioning
  • Messaging (Text and Email)
  • Translate other languages
  • Search and Surf Internet
  • File Manager (Advanced)
  • Play music (personal and internet stations)
  • Play and movies (personal and internet stations)
  • Show pictures
    • Multiple sources such as Dropbox, Google Drive and Local
  • Edit Photos
  • Vector Drawing (has own tools interface)

Ultimately everything should be voice driven. Voice commands are translated into a set of objectives. The user can see these objectives in simple command form and decide if it is acceptable. And that's key --everything should be scriptable.

I wonder if we will ever have anything like.

Written By
trans —

Html Follies

HTML Follies

DRAFT

Despite all its glories, HTML also has some serious follies. I've never thought highly of the W3C in general. They seem to be on a never ending quest to build standards at a snails pace, yet end up with a result that is barely an improvement over previous solutions --if not actually worse. And unfortunately it is too often at an intolerable level of new complexity. For some reason K.I.S.S. seems to have been left off their principles sheet. For some clear examples, consider [XMLST] and [XML-fo]. I think [Web Components] are the latest ranking member.

So let's go through some of the glaring issues with HTML, what would be an appropriate solution and perhaps make some suggestion for how to mitigate the issue while still under the yolk of current standards.

Headers Ought to be Titles

Presently HTML authors must handle the titling of sections as follows:

<section>
  <h2>New Chapter</h2>
  <p>...</p>
</section>

We assume here that the author was at least wise enough to put the header in the section and not above it. But why h2 (which is a common choice)? Why not h1 or h3? Well, there is no real reason. The document creator simply choose h2 because it was in suitable font size as setup in the CSS. And therein lies a clear failure of design. The element is not being selected semantically. A proper design would be:

<section>
  <title>New Chapter</title>
  <p>...</p>
</section>

Alas, HTML has no title element (outside of head). We could argue for a numberless h instead, but it amounts to essentially the same thing. Yet the term title is much more telling. Likewise subtitle is much more telling than any h2 or h3.

What to do now? Our best course of action is to always use h1 for titles and use h2 for subtitles. Headers h3-h6 should rarely, if ever, be of any use.

Padding Undermines Width

Technically this is a CSS issue, but these days it's not reasonable to separate it from HTML. The two are intractably linked.

Padding should not add width. If the width of an element is specified and a padding is added, then the inner width should reduce accordingly.

Thankfully this has been addressed in the latest HTML version. Hallelujah! ...

Open Semantics

Lastly, the ultimate end point of the development of HTML is to separate the markup of structure from that of the meaning of the tags. This is in fact what the W3C was attempting with XML and XSLT. But that approach failed. It was too complicated. A new approach using CSS instead of XSLT could do better.

<chapter>
  <p>...</p>
</chapter>

Chapter isn't a recognized tag. But the CSS could be used to identify it.

chapter { tag: section }

Thus the CSS informs the document to treat chapter tags as if they were section tags.

Current the W3C's solution to this is Web Components. And while some of the Web Components specification is needed for more complex problems. It is still has a piss poor interface, and is a complete overkill for simple cases like the one above.

Written By
trans —

Transpilers To Watch

Transpilers are the new Black (Coffee)

CoffeeScript is fairly ubiquitous technology these days --a transpiler that converts it's own, much nicer language into Javascript, which can then be run via Node.js or on the Web. But it's not the only fish in this sea. Here are three similar projects that look very promising.

Forml

A contemporary programming language for the discriminating programmer, intended to approximate the safety of Haskell and the expressiveness of Ruby. Should such an approximation turn out to exist.

NoFlo

NoFlo is a JavaScript implementation of Flow-Based Programming (FBP). Separating the control flow of software from the actual software logic. Helping you organize large applications easier than traditional OOP paradigms, especially when importing and modifying large data sets.

Opal

Opa is a full-stack open source web development framework for JavaScript that lets you write secure and scalable web applications. Opa handles all aspects of web programming: frontend (client code), backend (server code) and database queries are all written in one consistent language and compiled to standards of the web.

Written By
trans —

Revelation Knowledge

I just had a revelation about programming languages.

Coming into this project, my primary goal was to rate a Forth derivative, that added types, polymorphism and functional-orientation to the language. That goal is proceeded reasonably well. However, in the course of thinking about all the varying aspects of the design I hit upon a very interesting concept. I asked, "What does the computer know?" In other words, how are we really informing the computer about our systems as we program. It only stands the reason that the more the computer knows the more we easily we can communicate our intent.

So lets start with the bread and butter the algorithm, and seeing that we want to stay in the realm of functional-programming ultimately will strictly speak of functions. But to proceed, lets get our barrings by asking what other languages tell the computer about such these things, whether they be procedures, functions or methods.

Forth is probably the most basic in this regard of all languages, with the exception of Assembly. It doesn't tell the computer anything at all about a word (Forth's term for these things) other then the procedure by which it is calculated. Some Forth words provide terse stack diagrams but they are simply comments, ignored by the computer.

Other languages, like Ruby, are essentially the same as Forth. Ruby is a dynamic language, so it doesn't provide types for method arguments. To a limited degree names are provided for them, but they are promptly thrown away by the interpretor, so the computer is no more knowledgeable in that regard either. We do get one piece of extra information though. We can ask Ruby what the arity of a method is.

Python takes us a tad step further with regards the argument names. One can pass arguments to a method by name. That's an improvement, but that's about as far as it goes.

On the other side of things, there are languages like Java which are very strictly typed. But the types do little more than inform the compiler how to do it's job better. In the language itself we can't really ask about them. The names of the parameters are still ultimately disposable to the computer. (I wouldn't be surprised if there is some insanely complicate reflection library out there for Java that would make some of this possible, but regardless it is not part of the real language.)

Then there are languages like Haskell which are strictly typed and can tell us all about those types.

Haskell's about the state of the art at this point. There isn't a language I know of that informs the computer more about it's functions.

So then I asked, what would be a knowledgeable design such that the computer would know about these things we computers use day in and day out, and really be able to converse with us about them, so to speak. Well, lets break it down. What is there to know about a function?

  • Arity
  • Types of the arguments
  • Types of the return values
  • Name of the function
  • Name of the arguments
  • Definition of the function

Beside these, it may even behoove us to inform the computer about aliases for the various names, so it is easier to talk about these things for different programmers who might not be as with the code, or just to give us some programmer to speak more naturally, rather than always being so tortured to use the exact term.

Lets try and example, by writing out some pseudo-code for a function. Let's keep it simple, say, calculating the tip for a check.

function tip(amount, percentage)
  typeof amount Currency
  typeof percentage Float

  default amount $0
  default percentage 20%

  aliases percentage percent pnct pct
  aliases amount price cost

  definition amount * percentage

  return-type Currency
end

aliases tip gratuity

But notice we also said "tip for a check". That means something, so in our pseudo-code lets try to express that.

context tip check

aliases check bill

Okay. So what do we have after all that? Well, at the very least we have a way to speak to the computer about the various parts of our function.

tip.parameters  => [amount, percentage]
tip.return_type => Currency
tip.parameter(amount).type  => Currency
tip.parameter(pcnt).default => 20%

And so on. We can also call the function of course, but we could do so in unique way now. We can do it in an object-oriented way. After we do have a thing that is our function.

mytip = new tip
mytip.amount = $100.00
mytip.return  => $20

Now in the larger scheme of things, if that is all there were it might not get us too much further than the programming languages we mentioned above. But, we might now be able take all this new found knowledge granted to the computer about our program and start asking it to do some things we might normally do ourself.

For example, lets describe to the computer the concept of paying for lunch.

function pay(bill, tender)
  type bill Bill
  type tender Currency
  definition
    tender - (tally + gratuity(pcnt: 20%))  # this right here!
  return-name change 
  type change Currency

object Bill(menu-items, tax)
  type menu-items List(Product)
  type tax Percentage

function tally(bill)
  type bill Bill
  definition
     net = sum(bill.menu_items.each.cost) 
     net + net * tax
  aliases tally price cost
  return-type Currency

object Product(name, price)
  aliases price cost
  type name String
  type price Currency

Look closely at the line tally + gratuity. Notice we didn't say bill.tally + tip(bill.tally). Not I know my pseudo code doesn't really get us to the point where it's clear how we can be so concise. But I think the basic notion is there. The idea is that by explicitly defining the various parts of our model, drawing relationships between those parts via aliases then the computer itself can start to make the connections between the return values and the input parameters for us.

I suspect if this kind of approach is taken to it's natural conclusion, it should ultimately allow us to write much more reusable code, write that code in a much more natural and information rich manner.

Taking a step back, I believe it would be too much of undertaking for me to endeavor upon at this point. Nonetheless it does move me in a smarter direction with March --to make the language more knowledgeable.

Written By
trans —

Hardly Functional

Is Thinking Functionally a Black Art?

As I explained in a previous post, I have been writing a genetic algorithm to find the best layout of a keyboard based on a set of ergonomic principles. I decided to write the program in both Go and Elixir as a way to "learn me some new programmings languages for the greater good". Writing the Go program went very smoothly. I had a good working program in just over a day's work. The Elixir code on the other hand has taken me close to three days and I am still trying to work out issues. I attribute most of this long spell to the paradigm shift one must make when switching from a procedural to a functional way of doing things.

I have two examples from my work that I think demonstrate the difficultly of this shift.

Originally I had written the following code:

    def evolve(gen) do
      :random.seed

      best = 0
      size = 16

      pop = random_keyboards(size)

      Enum.reduce (1 .. gen), pop, fn(i, p) ->
        newpop = List.concat(breed(p), p)
        newpop = natural_selection(newpop, size)

        first = Enum.at(childs, 0)
        display(first)

        if best < first.score do
          best = first.score
          size = 16
        else
          size = size + 2
        end         

        newpop
      end
    end

It all seemed rather reasonable to me. Turns out however, Elixir thought otherwise:

variable best is unused
variable size is unused

My procedural mindset was geared toward the idea of using variables to track a state while iterating around in a loop. But for a functional language this is not the correct way to think about the problem. By reassigning best and size within the reduce block function I was in effect creating new variables, not reassigning the ones defined outside.

The solution, as is the case with many such procedural conversions, is to think recursively.

    def evolve(gen) do
      :random.seed
      evolve(gen, random_keyboards(16), 16, 0)
    end

    def evolve(gen, pop, size, best) do
      if gen == 0 do
        pop
      else
        newpop = List.concat(breed(p), p)
        newpop = natural_selection(newpop, size)

        first = Enum.at(childs, 0)
        display(first)

        if best < first.score do
          evolve(gen - 1, newpop, 16, first.score)
        else
          evolve(gen - 1, newpop, size + 2, best)
        end         
      end
    end

After working this out, a part of me has to agree the fanboys of functional languages. This does smack of an certain elegance that the procedure solution lacks. The solution is also efficient thanks to tail recursion, which avoids the usual call stack overhead of repeatedly calling nested functions.

But then there was the routine that I have yet to figure out how to do in a more functional way. To get the following routine to work I basically had to make Elixir behave much like a procedural language --running in a loop shuffling list indexes around.

    def cross(mother, father) do
      offset = 0

      if numeric_layout?() do
        offset = 9
      end

      n = number_of_actions(1)  # 0 or 1

      child_layout = Enum.reduce (0 .. n), mother.layout, fn(_, c) ->
        i = :random.uniform(length(c) - offset - 1) + offset
        l = Enum.at father.layout, i
        x = Enum.find_index(c, fn(q) -> q == l end)
        if x do
          swap(c, x, i)
        else
          c  # should generally never happen
        end
      end

      make_keyboard(mutate(child_layout))
    end

Perhaps it is inevitable that some procedural-esque code will always be present. In fact, that is the one caveat about functional programming that makes me wonder if maybe it's not quite all that. Some problems, that are relatively easy to solve in a traditional procedural language, seem markedly harder to solve functionally.

Written By
trans —

Should You Go

Should You Go?

2012-080-06 [DRAFT]

Preface

I've been working on a (somewhat esoteric) IME for Android over that last couple of years and recently I decided to automate part of my design process. I had been manually evaluating keyboard layouts for ergonomic quality. After expending so much time and energy into evaluating one layout after another, trying to ascertain which is optimal in my mind without coming to any firm conclusion, I decided something had to give. And so I did what a coder does best: I wrote a program to figure it out for me. The idea was straight forward. I would codify the principles by which an ergonomic layout is to be judged, assign a score to each of those principles, and then have the computer shuffle through every permutation to find the one with the highest score.

Thanks to its elegance and my high fluency in Ruby, I was able to whip out an initial working program in a few hours, and by the end of the day I was getting useful results. Unfortunately there was a problem. It was Way Too Slow. The first culprit, it become clear, was that a brute force search wasn't going to suffice. There were simply too many possibilities. To remedy that I added a simple genetic algorithm and seeded it with an initial population of layouts I already knew to be pretty good. That helped a lot, but it was still Way Too Slow. So then I decide to turn this new problem into an opportunity. And that's where this story gets interesting...

Learning Go

I've been wanting to learn a new modern programming language for a while. Having a relatively small yet substantial problem to solve made for the perfect excuse. After doing some research on Go, Julia, Haskell, Erlang and Elixir --all languages I expected to offer me a nice bump in speed (wait, isn't that just about any language compared to Ruby?) I narrowed my choices to Elixir and Go. I choose Elixir because it was a functional language but borrowed much of it's syntax from Ruby. That would make the transition easier. And I choose Go because it was a lower-level language, potentially offering the greatest speed gain. I've always hated C and I will never endeavor to be proficient in it. But Go presents a much more approachable way to program that is fairly close to the metal like C. So I am pretty tickled by its prospects. I dived into the Elixir first, but I will save my thoughts on Elixir for a future post. The rest of this article will cover my initial thoughts on Go.

Initial Impression

If I were to sum things up as concisely as possible I would say this about Go: "The really good thing about Go is that there is so little to it. The really bad thing about Go is that there is so little to it." That is very accurate. In Go, I had to write a function to scan an array of string for members. I stress "array of string", because, you guessed it, one would have to write a completely separate function for array of int. Go has no general functions. Every function is type specific to its very bones. Many a questions on the golang-nuts mailing list ends with the post "There is no general solution...". It really does seem crazy how many hoops one has to jump through to get things done. What would be a clear one-liner in Ruby, well, it will take at least a handful of lines in Go. In the future, I truly expect to hear old-time Go coders swapping legends about maintaining million lines of code. (Just like old times!)

But!!! (triple exclamation) There is a silver lining. As awful as all that sounds, the downright simplisticness(TM) of Go does have two really great benefits. First, there isn't much to learn with respect to the language itself. It literally takes a day to learn the basic constructs and to get up and running at a pretty good clip. Secondly, because every itty-bitty routine must be coded there is hardly any doubt as to exactly what the code is doing and there is plenty of room for optimization all along the way. This is one of the things that makes Go code so fast. Granted it leaves room for a poor coder to write less than optimal code too, but Go Trusts You. On the whole the extreme simplicity of the language is a net positive.

Beyond that, coding in Go was fairly comfortable. The syntax is actually harder to read than it is write, as strange as that sounds. After writing Go code for just for a short while, it starts to feel very natural, which probably accounts for why I was able to write my program in just over a day, despite never having written a lick of Go before. In particular I have to give props to Go's for loop construction. The whole for i, x := range anarray { ... } thing makes writing most loops a breeze.

Case-for-Visibility

One reason the code might be a little hard to read is Go's use of function name capitalization to "export" a package's public interface. There is a part of me that wants to agree with other that this is stroke of Genius. Many Rubyists feel the same about the capitalization of Class names. However, when ever I see capitalization being made significant in a language a little red flag goes off in the back of my head. Upon a more tempered consideration, it is perhaps easy to see why. The designers admit this is an issue for those who code in certain foreign languages. But they also say, "the case-for-visibility rule is unlikely to change however; it's one of our favorite features of Go." It would seem the designers might need a lesson on the Adrien Brody Rule. If it were only a convention it might be passable. But forcing it is really unacceptable for wider use as a global programming language. And personally, after the initial coolness wore off, I began to think it just looks bad. Alongside Go's CamelCase convention (and you kind of have to do that once you require capitalization), you end up with somewhat ugly, "Visually Basic" looking code. Clearly they missed the Ruby memo: When it comes to readability, foo.make_something_good beats foo.MakeSomethingGood.

Project Structure

Another significant issue with Go is its standard project structure. Essentially it pushes the one giant workspace approach to development (see http://golang.org/doc/code.html). That might make sense for a software development house, but for other types of coders that is really not acceptable. I sometimes have projects where the code is just an auxiliary aspect. The project could have presentations, spreadsheets, word documents, etc. If there is code associated with the project it needs to reside alongside these other documents and not be handicapped for doing so. Currently the only way I see for doing this is to adjust the $GOPATH every time I work on such a project. No fun. Hopefully there is a better way yet too be discovered, or that the Go designers will eventually provide.

Debugging

From what I thus surmise about the tools provided for Go, I have only scratched the surface of what is available. But I can say that one the best things about working with Go were the very clear and directory error reports. By default, at least, Go doesn't fill the screen with stack traces and code snippets. Rather is simply states the problem in plain English and give the file and line number where it arose. The clarity of the error messages made it very easy to correct every issue detected by the compiler.

Documentation

With regards to documentation, I have to give Go a push. What is available is still quite weak. However, given how new the language is, the current state of documentation is actually rather commendable. With its growing popularity, clearly this is a situation that will improve rapidly.

Einstein had a saying, "Everything should be made as simple as possible, but not simpler". In some respects, such as the generics issue, Go has made things too simple. If Go's designers ultimately find a way to support generics --and by all accounts they are at least open to the prospect, the language would become much more accessible. And some of this critique will be mitigated. But despite it's over simplification, it beats the pants off the converse issue over complexification, which we too often find in other languages.

Go is the best new language to come along since I learned Ruby in 2002. I think anyone still coding C when it's not absolutely necessary, is just a glutton for punishment. I highly recommend you Go, and Go now.

Written By
trans —

Older ›