In Progress – 2016+

The past year has been fairly quiet for me, in terms of Smalltalk/Pharo projects. I started a new job at MIT CSAIL working for the Solid project (repo), creating a platform for radically decentralized data-centric applications. This has meant a lot of learning and unfamiliar territory — RDF and the various components of the Semantic Web tech stack, learning to write isomorphic Javascript (same codebase on Node.js/server and the client/browser), and diving deep into cryptography and decentralized authentication.

Some notes from the past year:

  • Been doing a lot of RDF work, as it is central to the current Semantic Web community. And while I haven’t found any RDF parsers in Smalltalk or Pharo ecosystems, I think the JSON-based JSON-LD RDF serialization format might be the easiest route for getting some basic RDF capability into Pharo.
  • There are some fantastic JSON-based standards out there, including JSON Schema, the JOSE suite, which includes JWT/JSON Web Token and the corresponding JWS for signing and JWE for encryption standards. My colleague and I have been working on an ecosystem of isomorphic Javascript libraries to support the JSON Schema / JOSE stack. I am of course curious about what is the state of the art in JOSE-related support in the Smalltalk world, but have not researched it yet. Definitely something to investigate.
  • The state of the art in decentralized authentication is the OpenID Connect series of standards, based on trusty old OAuth2. Just like with JOSE, I wonder if there are any OIDC libraries in Smalltalk (I know there are at least a few for OAuth2?).
  • I’ve signed up for the upcoming Camp Smalltalk 2017 in Raleigh/Durham – very exciting!
  • The updated book, Pharo By Example 5 has been released recently! I am incredibly proud to have helped editing it!
  • I am still intending to continue working on the Phriak database driver, and at least bring it to Riak 2.0 feature parity. I wonder if the ProtoBuf library for Pharo is in usable condition by now?

In Progress

Some Smalltalk-related things I’ve been involved with lately:

  1. Phriak – a Pharo Smalltalk driver for the Riak distributed NoSQL database. (The project is also mirrored at SmalltalkHub at http://smalltalkhub.com/#!/~gokr/Phriak). I’m the current maintainer of the project, and have gradually been adding new features to get the driver to Riak 1.4 feature parity. Riak 2.0 feature support (full text search, server-side data types, etc) coming up next!
  2. I’ve been helping with editing of the Updated Pharo by Example book, which updates the original PBE book to work with Pharo version 4.
  3. Earlier this spring, I’ve watched with interest the release of Pharo 4 (and even submitted bug reports and tested fixes).
  4. This past weekend, I attended Camp Smalltalk Ottawa 2015. Some great presentations (Reg Krock is working on some interesting projects, like hooking up the Oculus Rift API to Smalltalk, as well as a custom Smalltalk virtual machine that runs on Android), and a good discussion on promoting Smalltalk at large.

Installing Pharo on the Server

Table of Contents:

Step 1: Set up your hosting server / OS – Ubuntu 12.04 LTS x64

For hosting, I went with a Digital Ocean VPS instance. The amount of RAM, disk space, bandwidth and processing power you can get for $5 to $10 US a month can’t be beat.

With this host, you select your operating system up front, and they create it from a ready image in a very short amount of time (with my previous host, it booted up to “bare” hardware with an Ubuntu install CD in the emulated disk drive, and so installation took a while).

OS-wise, I went with Ubuntu 12.04 LTS x64. Ubuntu because of familiarity and ease of dev tool installation, 12.04 LTS (Long Term Support) for its mix of newness, stability and long term security patch support, and the 64-bit version because the Riak db only runs on 64 bit systems, and that’s what I want to use for my Smalltalk persistence layer (via the Phriak library).

I’m pretty sure you can make all of this work on CentOS / RedHat, but I’m less familiar with those distros, and plus going with Ubuntu meant I could make use of the nice pre-packaged Pharo VMs, as you’ll see below.

Step 2: Install the Pharo VM

There are several ways of installing Pharo on a Linux server — see the Installing Pharo in Many Flavors post for a discussion.

However, the apt-get install method on Ubuntu is the most straightforward for beginners. (A huge Thank You to Damien Cassou and Nicolas Petton for making available the Pharo Ubuntu packages! These make server-side installation much easier.)

For instructions, I went with the recommendations from the Pharo on Ubuntu guide.

Enable the add-apt-repository command:

sudo apt-get install python-software-properties

Add the Pharo PPA (custom package archive):

sudo add-apt-repository ppa:pharo/stable
sudo apt-get update

Install the server-side version of Pharo. (Meaning, only pharo-vm-core, not the pharo-launcher:i386 package).

sudo apt-get install pharo-vm-core

You can see all of the files installed by pharo-vm-core via:

sudo dpkg -L pharo-vm-core:i386

The binary that you will be executing will be in:

/usr/bin/pharo-vm-nox

Note: -nox means “No X Windows”. Meaning, this is a server-side pharo, without any GUI components.
Also, you can view the help files from the command line:

man pharo-vm-nox

Step 3: Upload Your Seaside Image

Now that Pharo is ready to go on your server, it’s time to upload your deployment image.

First, locate your deployment image on your development machine.
On Mac OS X, the image is located in:

[Seaside install/download folder]/Seaside-3.0.7.app/Contents/Resources

Copy the image to the server using scp. For example, I would use:

scp ~/Seaside-3.0.7.app/Contents/Resources/Seaside.1.image my-username@xx.xx.xx.xx:/home/my-username/

Step 4: Start Pharo

Finally, to launch the Pharo VM and point it to your image:

pharo-vm-nox ~/Seaside.1.image

This will load the image from my user home, and start the Seaside web stack listening on port 8080, by default.

Of course, this is merely the basic concept of launching Pharo, for tutorial purposes. For a production application you would want to ensure that Pharo is started via a system service. In addition, you would want to put a web server such as Apache or Nginx in front of it, for security and load balancing purposes.

Packaging Your Squeak/Pharo Image for Deployment

Table of Contents:

There are two schools of thought, as far as preparing a Smalltalk image for deployment to a remote server:
1) Start with a minimal clean image, and only load the libraries and classes that you’re actually using to run your application.
2) Take your development image, and strip out unnecessary applications, GUIs, and the development environment itself

The first method is slightly more difficult but very rewarding, especially if you’re going to go the Continuous Integration route (via Hudson/Jenkins), because it requires you to know exactly which packages you’re using, what they depend on, the load order, and so on. We will explore this approach in later posts.

For this tutorial, though, we’ll use method #2, with help from the Preparing for Deployment chapter of the Seaside book.

Things to Remove for Deployment

  1. The Developer Toolbar
  2. All the default applications (except our Hello World one).

First off, what is the Developer Toolbar? It’s the dev mode menu that Seaside automatically inserts at the bottom of your applications, to help you with configuration, session resetting, and so on, and it looks like this:

seaside_dev_toolbar

You definitely want to disable it when deploying your application to a production type server. Fortunately, the code that unregisters all of the extraneous default applications (see below) also removes the dev toolbar.

First, unregister all applications with Seaside (to view the list of the applications that are installed, pull up http://localhost:8080/browse)

WADispatcher default handlers do: [ :each | 
   WADispatcher default unregister: each ]. 

Next, re-register our Hello World app, since that’s what we actually want to deploy:

WAAdmin register: RHelloWorldComponent 
  asApplicationAt: 'helloworld'

Now, double-check that our application is running on our localhost server at http://localhost:8080/helloworld, and that the others (such as /config or /browse, aren’t).

The last step is to save the image. I recommend the Save As route, so as not to overwrite your development image — left-click on the Pharo desktop to bring up the World menu, select Save As:

seaside_world_menu
And pick a name for the new image, such as:

seaside_save_image_as

Now your image is ready to be shipped to a remote server for deployment.

Definition of Good Software Developers and Software Architects

Alex has a very interesting post up, What the good developer and the good software architect really are.

His definition (explained at length in the post):
A “Skilled computer user is a person who can accomplish most of their goals using computer and software installed on it by himself“.

A “Software Developer is a person who can make another person be able to accomplish their goals using new software they made“.

A “Software Architect is a person who can make another person be able to accomplish their goal as a Software Developer“.

I think this is a helpful definition, both in our roles as software developers (to try and be clear about what we want our users to be able to accomplish, and to view their failures in using our software as calls to our own improvement) but especially as software architects (we need to have clear goals, as architects, about what kind of capabilities we want our software developers to have, and to measure our success by theirs).

Interlude: How to get a remote server (for Seaside deployment)

Table of Contents:

For the purposes of this tutorial, I’m going to walk you through setting up Pharo on a remote Ubuntu Linux server. This, of course, requires first getting such a server. The easiest (and cheapest) option that I’ve found is to get Squeak/Pharo running on a low-cost Virtual Private Server (VPS).

Here’s what I want you to come away with: Hosting for Seaside is fairly easy, and within your reach.
And, it is no different than hosting options for other (non-PHP) languages, such as Ruby, Python or Java. If you’re a hobbyist developer, or just want to test it out, there’s free options available. If you’re a pro, you can get a (virtual) remote server with full shell access for very little, for just $5-20/month, USD, or an actual physical server for about a hundred a month. Pair that with basic Linux sysadmin skills, and you’re good to go.

Low Cost Options – Virtual Private Servers

Getting yourself a Virtual Private Server hosting account is the easiest and cheapest way to get started with deploying your apps (in any sort of dynamic language). This is what I use, myself, at the moment.
Advantage: Low cost, you have full root access to the OS
Disadvantage: Self-managed (you have to provide the system administration), low disk space (compared to dedicated servers — VPSs have from 10-50Gb of disk space), slower (than dedicated physical servers)

For hosting, I went with a Digital Ocean VPS instance. The amount of RAM, disk space, bandwidth and processing power you can get for $5 to $10 US a month can’t be beat.

I polled my web app developer friends recently, as to what sort of hosting service they use (they are mostly Ruby and Python developers). They also use VPS hosting, with either Digital Ocean, or Linode (there was one person using Slicehost, but they were switching to Linode). Out of those, Digital Ocean has the best value, although Linode has is more established and highly regarded for business use.

Low Cost Options – Cloud Hosting

Pharocloud looks very promising. They are attempting to provide the same level of ease and convenience for Smalltalk developers as Heroku does for Ruby/Rails developers. The important feature here is that they also provide persistence services (Postgres, etc) for your Pharo/Seaside/whatever images.

Amazon Elastic Compute Cloud (EC2) For not much more cost per month than a medium-size VPS, you can also deploy your Seaside application to the cloud, which gets you load balancing, unlimited scaling capability, and more.
Deployment to EC2 – Resources:

Medium Cost Option – Dedicated Servers

If you need fast performance, and lots of memory and disk space (and you’re a professional developer or a small company), you have to go with an actual dedicated physical server at a colocation facility.
Fortunately, these are still fairly inexpensive (compared to what they used to cost even a few years ago). You have two options:
Unmanaged Servers (they give you a physcal box and remote access, just like with a VPS, and you provide the system administration). DedicatedNow has unmanaged servers for $99 USD/month (this includes a 2-core CPU with 2Gb RAM and 2Tb disk space), and Westhost has a slightly less beefy server (with 500Mb ram) for $95/month.

You can also choose Managed Servers, where the host provides you with a box and performs system administration for you, guarantees your uptime and manages your security. DedicatedNow has a managed server (with 4Gb RAM!) for $200 USD/month.

Free Options

If you just want to try out the Smalltalk/Seaside deployment process and you do not want to pay, or if you’re developing a free service, you have a number of free options.

Seaside Hosting offers free 256MB hosting partitions for non-commercial Seaside applications (see the chapter on deploying to SeasideHosting of the Seaside book, and the Seaside hosting Pharocast). This is where the official Seaside site is hosted, for instance.
Advantage: It’s free, and they’re knowledgeable about Seaside.
Disadvantage: For non-commercial apps only, 256MB disk limit, no shell access or access to databases (so, you basically store your data in the image). See also this StackOverflow question, Can I host a SandstoneDB on SeasideHosting.st? – and the upshot is, no, you can’t at this point.

Free Shell Access
There is also a number of places that provide you with free hosting accounts that you can SSH into — see the Free Shells List or just google for ‘free shell’ access. For free, these are obviously going to be small and underpowered hosting accounts, with not much memory, diskspace or bandwidth provided. But, they are full on development environments, with database access and everything and you can get Squeak/Pharo to run on most of them.
The only one of these free shell services I have experience with is the SDF Lonestar organization – and hey, it does what it says on the box.
Advantage: Free. Full SSH access, databases
Disadvantage: Slow, resource restrictions (RAM, disk space, bandwidth), and there may be issues with Terms of Service restrictions

Open Source/Community Applications
One more thing. If you have no budget, but you’re developing a free and helpful service that might benefit other people, don’t despair. Develop it, and put it up on a free hosting service above for beta testing. When you outgrow free hosting and need more power (or disk space, or bandwidth)… you should just ask. Seriously, if you’ve come up with really good software that benefits people, the community can probably find hosting for you. Ask on the mailing lists (Seaside, Squeak, Pharo), ask individual developers or universities… Doors will be opened to you.

News Roundup: Social Code Commenting, User Management, and Java Interop

Couple of interesting items from the Pharo mailing list.

Comment of the Day Contest

In a thread titled The Comment Game – MMORPG for real, Laurent Laffont proposed a game. Each day, an un-commented class would be picked from the Pharo class hierarchy, and people on the list would propose comments for the class (which would then be added to the next release).

This is a great idea, not to mention hilarious. Comments (especially class comments) are worth their weight in gold, and this is an easy and low-key way to slowly improve the quality of the codebase. Not to mention harnessing the wisdom of the crowds, etc. Several classes have already been commented: MCFileRepositoryInspector, HTTPClient, and more.

The results of the daily commenting contests are collated at: Pharo Wiki – CommentOfTheDayContest

JNIPort, a Java interop library, announced for VisualWorks, Pharo and Squeak

Satuday, the release of JNIPort 2.0 was announced by Joachim Geidel. JNIPort is “a Smalltalk library which allows Java code to be invoked from
Smalltalk”. This is excellent news – the more interoperability options the Smalltalk ecosystem has, the better. I look forward to checking out the library.

New Authentication/Login/User Account Management package for Seaside

Earlier this month, Tony Fleig announced a new package, TFLogin to handle some common web app tasks – user authentication, registration and account management. This is incredibly important, and I will be reviewing this package and discussing Seaside user management options as soon as I’m done with the Remote Hello World post series.

Thanks Tony!

Arlo Guthrie and the origins of the Collection protocol

A bit of history to make you smile:

Rick DeNatale at Talk Like a Duck writes about
How Arlo got injected into Ruby [via Smalltalk].

During the interview he [Dan Ingalls] was asked about the origin of those enumeration methods of the Smalltalk collection classes. Alan Kay had told the interviewer that they had come from a song. At first Dan didn’t remember this but then remembered that there was a song which had a string of words like inject, select, detect etc. As far as I recall, though he didn’t name the song.

But I recognized it right away, here’s how “Alice’s Restaurant Massacree” transitions from the littering trial to the draft:

Came to talk about the draft. They got a building down New York City, it’s called Whitehall Street, where you walk in, you get injected, inspected, detected, infected, neglected and selected.

So that’s how the enumeration method names came into Smalltalk (and, later on, into Ruby)!

Creating a Seaside “Hello World”, localhost

Table of Contents:

Note: This section assumes that you’ve mastered the basics of creating new classes and methods in Pharo Smalltalk. If not, I recommend you glance through the Paro By Example, or my own previous post on Creating Classes and Methods in Pharo.

Note 2: See also the Getting Started (Pharo and Squeak) section of Dynamic Web Development with Seaside for a similar “getting started” tutorial, with screenshots.

Time to create our first “Hello World” app in Smalltalk/Seaside. Here’s the plan:

  1. Create a new Category and component (read: class) for our app.
  2. Create a method to output ‘Hello World’
  3. Register the component with Seaside, so it can start routing requests
  4. Profit! (That is, load up the page in the browser, and behold Hello World)

1.a. Creating a Seaside Component – The How

Creating a Seaside component is easy. First, create a Category for this tutorial – something like ‘RHelloWorld-Core’. Now, inside it, create a class (a subclass of WAComponent) called, for example, RHelloWorldComponent. Like this:

WAComponent subclass: #RHelloWorldComponent
	instanceVariableNames: ''
	classVariableNames: ''
	poolDictionaries: ''
	category: 'RHelloWorld-Core'

That’s it. You’re a third of the way done (now we just need to create a method to output ‘Hello World’, and register the component with the framework, so it can start routing).

1.b. Creating a Seaside Component – The Why

So, what is a component? A Seaside component is a subclass of WAComponent (the “WA” stands for “Web Application”, incidentally). Great, but what is it really? To answer that, we need to briefly discuss how you write web apps in Seaside.

Think of a component as a “smart view”. What about Models, though? And Controllers? Well, components that are complex enough to need models, have models. That is, Models are also a concept that Seaside uses. Since we’re just writing a static “Hello World” app, we don’t need a model, we’re just going to output a literal string directly. What about Controllers, though? Don’t we need a Controller? Well, no. Not as such. Part of the answer is – Seaside (and the programming language itself) IS the Controller. It takes on the traditional role of the controller (routing requests, calling the right objects and methods, managing sessions and marshalling parameters). Seaside also has Tasks (subclasses of WATask), which are view-less control and routing objects that manipulate other components, a job traditionally reserved for Controllers in the MVC paradigm. But again, we don’t need Tasks for a Hello World tutorial.

The easiest way to understand Seaside web development is to think of it in terms of traditional desktop GUI development. Say you open up a desktop application, like Firefox or Outlook Express. You see a window, and within it, panes and toolbars, and within those, buttons and lists and other GUI widgets. In Seaside, the application/”window” is a top-level Component, which displays sub-components (toolbars, navigation menus, panes). In a desktop app, the components keep track of their own states (and have models if need be), and have Events/Actions defined for when you interact with them (click on a button, scroll the scrollbar, and so on). Same thing in Seaside. So a Seaside app is “just” a nested collection of components, which keep track of their own states, and which register events or actions that are performed when a user clicks on a link or selects a value from a pulldown, and so on.

If that’s too confusing, or if you haven’t done desktop GUI programming, don’t worry about it. In Seaside, you’ll be dealing with Views, Models when you need them, and occasionally view-less controller-like Tasks.

2. Create a method to output ‘Hello World’

Ok, let’s create a method in our RHelloWorldComponent class. (You can either put it in the default --all-- method category, or, more correctly, create a rendering method category, and add the method there).

renderContentOn: html
	html text: 'Hello World!'.

3. Register RHelloWorldComponent with Seaside

Lastly, we need to register our component with Seaside (specifically, the Web Application Admin class), so it can start routing requests to it.

Execute the following snippet of code:

WAAdmin register: RHelloWorldComponent 
	asApplicationAt: 'helloworld'

Hold up. How do I execute code in Squeak/Pharo?
You can execute code from any code editor or workspace in the Pharo image. Traditionally, snippets of code (or “incantations”) are executed from a Workspace window, which just looks like a blank text editor window. To open one, click anywhere in the background of the Pharo window to bring the the World Menu, and select Workspace. You’ll see a blank text window. Now you can type arbitrary code in here, select it with your mouse, right-click on it and select Do It (d). The keyboard shortcut is Meta-d, of course.

Ok! Now, when you access the /helloworld url on your web server (in our case, running it by default on the desktop, it will be http://localhost:8080/helloworld), Seaside is going to locate the RHelloWorldComponent (since that’s what we registered as the top-level component of our application up above) and execute the renderContentOn: method.

Test it out – load it up in your browser, and you should see

Hello World!

displayed on a plain white html page.

Coming up next: Obtaining access to a remote server, installing Pharo on it, and packaging up your app for deployment.

A Secret: Passion and Your Choice of Web Framework

I will tell you a secret. I am passionate about Smalltalk and the Seaside web app framework. Not just on an intellectual level, “oh hey this is pretty cool, it’s got some good ideas”. But on a fire-in-my-heart level, “this is profound and important technology” kind of stuff.

Why? Good god, why? Why should a choice of language, or especially a choice of which web app framework to use, matter? Because, I believe that:

Your choice of app framework deeply affects your development speed, and your overall level of courage.

Choosing the right framework means you will finish your projects faster, and you will dare to include better and more audacious features in it.

Life is very short. You have a limited amount of time in which to realize your dreams, in which to do your projects. If you’re like me, you probably have dozens and dozens of ideas for apps, sites, portals, that you want to implement. Most of those ideas will suck. But you don’t know which ones, you won’t know until you try them. Your goal, in this life, is to iterate faster. To fail more, to fail quicker, until you get to the brilliant ideas and brilliant projects that can change the world, or at least make you a dollar or two.

I want to be able to code at the speed of thought. At the speed of dreaming. Or rather, at the speed of essential complexity, without the overhead of the accidental. See, there is a notion in computer science of Essential complexity versus Accidental complexity. Essential complexity is “Hmm, I’m trying to solve a difficult real-world problem that is complex and messy”, and accidental complexity is “why the hell can’t I easily do this simple thing in my framework?!”. And I believe that while the first will never go away, there is still too much of the second, of the overhead, of cruft, in the world of web frameworks.

Based on my experience, I believe that the Seaside framework is the most powerful and flexible one, that has the potential to come closest to “development at the speed of dreaming”. It is not quite there yet. But it can be, with not too much more effort.

Now it’s time for a confession: the fastest-to-develop in, most prototype-friendly framework that I’ve had the pleasure to work with so far is not Seaside, nor is it Ruby on Rails (which has other reasons to recommend it). It’s a relatively unknown (dare I say underdog?) Python framework called web2py, headed by Massimo Di Pierro. Its ease of use, quality of documentation and community support, and the amount (and integration) of out-of-the-box features made (on large project with a small team) development quick, easy and secure. I am seriously grateful to Massimo and the web2py community for raising my standards of what a web app framework should do.

Then what am I doing here? Why isn’t this blog called “web2py Zen”? Because, I believe that the features that make web2py fantastic (like comprehensive documentation, out-of-the-box authentication and user management, ease of use and diversity of persistence options, amazing error logging capabilities, etc) can be added to Seaside – with some effort, but it’s not impossible. Whereas the key features of Seaside (the continuations, the reuse and embedding of components, Halos and the debugging process) cannot be easily (or at all) added to web2py. Smalltalk (and Seaside) is fundamentally “upstream” to Python, in the sense that Paul Graham writes about.

So, Seaside could be my ideal web app framework. It is built on some amazing technology, and has features unsurpassed by any other framework at the moment (and has plenty of drawbacks, sure, but none unfixable). It’s going to take some work (and that’s part of what starting this blog has been about, for me), but it also has a vibrant community around it that I think is up to the task.

One more thing. While I believe that the choice of frameworks is important, and that some are, objectively, better than others (for certain goals), I don’t want the arguments over which one is better to be based solely on personal opinion. I want the frameworks to prove themselves, to pull their own weight. What I really want to see is studies, and competitions between frameworks. I want web framework olympics, of the kind that Plat-Forms is trying to do (only that’s in Germany – I would love to see more US-based ones). I want to see evidence, and experimentation. And will strive to provide some of my own.

Am I advocating that you choose one framework (that you think is best) and really focus on it, practice the hell out of it? Yes. But also: You have to try all the other frameworks. You have to periodically re-evaluate your choice. If some other framework is doing something fundamentally right, and you can easily add it to your framework, do it. If not, if they’re doing something right and profound and you can’t add it to your framework (perhaps because of language limitations), perhaps you need to reconsider your allegiance.

In any case, whatever your favorite language: Choose the right app framework. Love it, improve it, practice the hell out of it. Re-evaluate your choices periodically. And iterate faster.