Good Stuff

Lift Basics and Broad Shoulders

Wed, 08 Feb 2012 10:58:00 -0800

The Lift community is amazing. It's a collection of more than 3,000 people building amazing apps with Lift.

The Lift committer group is amazing. It's a collection of more than 50 people who put time and effort into writing the code in Lift and more importantly into creating an excellent, supportive environment in the Lift community.

Between the community and the committers, the shoulders that support Lift are indeed very broad and very strong.

I am honored and excited that Franz Bettag and I will be helping to broaden the knowledge, involvement and support of Lift by offering a 1 day Lift Basics course at Skills Matter in London on March 15th. This will offer Franz the training necessary to teach "David Pollak's Lift Basics" to audiences in Germany (in German) as well as other places.

More importantly, it's yet another data point that demonstrates the amazingly broad shoulders that support Lift. And stay tuned... there will be more announcements that underscore just how broad and strong the support for Lift is.

Permalink

Institutional knowledge is gold

Tue, 07 Feb 2012 11:22:20 -0800

Nick Cronin wrote a very interesting piece on how technology enhances labor mobility: http://techcrunch.com/2012/02/04/labor-efficiency-the-next-great-internet-dis...

I have a different perspective on labor mobility... or more precisely, wether I as an employer would desire fungible talent. I don't.

Putting aside the functional skills that a given person has, there are important social skills that take time to assimilate. Getting communications and interactions and understanding right takes a fair number of interactions. It's not just something that happens instantly (okay, sometimes you find someone who "speaks your language", but that's not the norm, it's a glorious, but rare, event.) It takes time to cultivate a shared language, a shared set of norms, a shared set of expectations, and only then, can someone work best for a given organization.

The requirements for shared language (my shorthand for the broader set of social and cultural understandings) is necessary for a worker to work out well within an organization. For example, there are some people who won't show their code until, in their minds, it's perfect. Those people may thrive in one organization but utterly fail in another. This is also true of people that code incrementally (lemme toss this out, get some feedback, and try again.) In both cases (and granted these are extremes), different organizations will do better or worse with each type of developer.

But, you say, we can settle those kinds of things in an interview.

No, no you can't. Most organizations use similar words for describing their expectations and norms. Lots and lots of organizations look for "Rock Stars" and "Ninjas" and are "Agile" and the like. Yet, those words mean different things. I've seen companies that have 45 minute "stand up" meetings where the director of engineering gives out daily assignments. I've seen organizations that advertise for "Rock Star Coders" and are frightened and institutionally reject folks that I consider merely good coders because they are so much better than the existing pool of developers that hiring them would cause more friction than the value of the additional good code. It's only in the exploration of the relationship that the parties can come to learn about each other and determine if they have a shared language.

Yes, you say, that's all good and fine for "knowledge workers" but what about the replaceable humans who sling hammers or assemble cars or take care of our lawns, etc.

The vast majority of work that humans do requires judgment and understanding priorities and balancing those priorities. The folks flipping burgers at In 'n' Out make lots of judgments. The folks at McDonalds try to remove as many judgments as they can and they will do best hiring people who do not thrive on making decisions. I'm betting there's not a lot of cross-trading between folks who are successful and McDonalds and In 'n' Out.

In fact, I think one of the things that cost Detroit its leadership in the 1970s (leadership that it has yet to regain) is the view that the assembly line workers were fungible, not smart, not valuable. The mentality led to management/labor friction, anger, waste heat, and ultimately lost leadership. If GM and Ford and Chrysler has viewed every member of the company and the supply chain as a valuable contributor to the creation and delivery of products that make buyers happy, we would live in a different world. But I digress.

Yes, sometimes you want to have external professionals rather than have them on your staff. Lawyers are one perfect example. Two of many favorite corporate lawyers are Jon Gavenman http://www.cooley.com/jgavenman and John Hession http://www.cooley.com/jhession. I've interacted with a lot of transactional guys in my life, yet these two stand out as working well for me. That doesn't mean they would work out well for everyone. They each speak the same language that I do, but that doesn't mean they speak the same language as everyone. Neither of these guys are interchangeable with any other lawyer.

But the bottom line for me is that institutional knowledge is gold. It's the core value of any company. That institutional knowledge is not simply knowing the problem space, but also the institutional knowledge of the values of the company, what the priorities of the company are, and how the company deals with adversity. The only way to foster that institutional gold is through long term relationships and the idea that any worker, any member of the team is fungible reduces the value of the business.

Permalink

The transition of scala-tools.org

Fri, 03 Feb 2012 11:53:49 -0800

It's been a little slow in coming (those ship dates always slip), but the Sonatype folks will be taking over the hosting of Scala related artifacts from scala-tools.org.

Currently, Sonatype is rsyncing the entire scala-tools.org repository so that anything published to scala-tools.org will be mirrored up to Sonatype.

We have transferred the LDAP information for all the scala-tools.org such that you will be able to publish directly to Sontaype's servers.

In the next week or so, we will be pointing the DNS for nexus.scala-tools.org and other domains over to Sonatype's servers such that existing projects will be communicating directly with Sonatype's servers (we were hoping to do that before the announcement, but, well... things happen.) We will also be 301-ing all incoming requests for artifacts to Sonatype's servers so that existing projects with existing configuration files will be getting the latest and greatest from Sonatype's servers.

Between now and mid-June, scala-tools.org will point to Sonatype's servers. Once the redirects are in place and periodically thereafter, I will be making reminder posts to point configuration files to Sonatype's servers. I will also send out periodic emails to the folks who have been publishing to scala-tools.org to update their configuration files to point to Sonatype's servers and to remind their communities to update configuration files to Sonatype's servers.

In mid-June, scala-tools.org will no longer redirect to Sonatype's servers.

I'd like to thank the Sonatype folks for being so helpful about transitioning the repository from scala-tools.org to the Sonatype servers. I'd also like the thank Derek who came up for air and helped out with the LDAP stuff.

More information and technical specifics as they become available.

Permalink

DPP's Lift Office Hours Monday February 6th

Fri, 03 Feb 2012 09:46:36 -0800

David Pollak will be available for Lift Office Hours to answer Lift-related questions either in person or on Skype from 11am to 3pm Pacific Standard Time.

Physical Location:

541 8th Street

San Francisco, CA 94121

Skype: lift-office-hours

Drop on by, give a call, I'll be glad to help!

Thanks,

David

Permalink

Why Visi is written in Haskell

Tue, 24 Jan 2012 11:01:49 -0800

Visi and Visi.Pro started life based on ideas I had when I was working on Apache ESME. One of the key features of ESME, a feature I wrote in a couple of hours 'cause I was bored, was "Actions". Actions allowed a user to declare what to do with messages that arrived in their timeline. You could filter messages (only show 20% of the messages from my overly chatty co-worker). You could email messages (if it's an @ message to me and it's from my boss, email it and text me.) You could cause arbitrary actions by generating HTTP POST requests to external services based on certain criteria (post to Twitter all messages in my timeline with the #Lift tag.) Actions made ESME different because it allowed a user (not a programmer) to define what to do as stuff flowed into their timeline. ESME was an SAP ecosystem project and we imagined that items in your timeline would include events generated from ERP systems: shipping exceptions, sales events (closed an order, received payment, etc.) and using Actions to manage the events seemed like a really cool idea.

ESME was and is based on Scala and Lift. I do not think ESME could have been written in such a short time with such a small team back in 2008 with any other technology.

I wanted to start an exploratory project within the then incubating ESME project and see what we could do to create an event based system and event-based grammar for defining complex systems. The Apache overlords where not happy with the exploration or the fact that I wanted to explore using Git on GitHub, so I shelved the project, but not the idea.

For the next couple of years, the idea of an "event processor in the cloud that could be managed by anyone" rattled around in my head. I've got pages in notebooks here and there about how the system could work, what the revenue models were, etc. In June 2010, I wrote a complete business plan for a cloud-based event management system. Having revenue from users to pay for the service plus a cut of revenue from data feed providers (a model influenced by Apple's App Store) seemed to make sense. At that time, I envisioned the system was built on Lift and Scala.

I got side-tracked in the summer of 2010 assembling a team and writing a business plan, etc. and I put the cloud event thing on the shelf. I ultimately did not join the team that I assembled and by the time 2011 rolled around, I was up to my eye-balls in family stuff. My wife went away to run a trial for 4 months (she won $310M for her client) and my mom passed away. Sometime in the Spring on 2011, I bought an iPad for the kids so they could watch Sponge Bob and other stuff on the flights down to visit my wife in Southern California.

The iPad changed my world-view on computing devices. The iPad is, in my mind, the first material change in the way people use computing devices since the Apple ][. Everything between the Apple ][ (the IBM PC, the Mac, laptops, netbooks, etc.) were incremental advanced on the technology that preceded them. But the iPad was different.

It was in May, after I had spent a week with an iPad and no other computing device, that I realized that PCs and the Web were legacy. This isn't "disappear tomorrow" legacy. This is "in 15 years, most people will not starting new web app project" legacy. Just like client-server is legacy, but there's still plenty of Windows-specific client-server code being written.

It was in May that I added "and a client component that runs on the iPad and iPhone" to my "event processor in the cloud" idea. Having a client component, a beautiful, rich, interactive, tactile, wondrous client component because the driving force in my thinking. Chatting with folks like my friend Bill Fernandez (Bill also did some amazing work on ESME) about the idea drove my thinking to making a programming system like HyperCard that was useable by lots of people. Given my spreadsheet background, especially my real-time spreadsheet background, gave rise to ideas around how to imagine a language for describing event processing and user interactions and such. Those ideas crystalized into Visi and Visi.Pro.

One of the gating factors with iOS, the operating system on iPad and iPhone, is that there's no JVM or anything JVM-like. There's only native code (well, there's LLVM as well, but the Scala-LLVM project is not that far along.) So my choices of language for the project were reduced to things that compiled to native code. I no longer have interest in manually managing memory and I like functional programming... it's how my mind has been re-wired. Plus, I'm a statically typed kind of guy... that meant that my language choices boiled down to OCaml and Haskell. Both had ARM cross-compilers and both were mature languages with mature ecosystems. Sadly, because Scala is still the language I'm most comfortable with and the language I'm most productive in, I could not use Scala for Visi.

OCaml is arguably the easier choice for Visi. It's conceptually a lot closer to Scala than is Haskell. The iOS cross-compilation tools for OCaml are much more mature than those for Haskell (although Stephen Blackheath's GHC-iPhone is coming along very nicely.) Interacting between OCaml and ObjC is much easier than any Haskell/ObjC bridge. Unfortunately, as many times as I try and as much excellent encouragement I get from Paul Snively, I just can't get into OCaml. I don't know why and it's purely subjective and it's probably something broken in my brain... but OCaml always feels like an ill-fitting scratchy wool sweater to me. This is not meant as a criticism of OCaml, but I don't like eating brussel sprouts either... it's just a taste thing.

So, that left Haskell for me. Haskell has some really helpful stuff. The Haskell parser-combinator library is excellent and mature... helpful for writing a language. Haskell's runtime is amazing in the way that computations (threads) can be suspended nearly costlessly. This makes as lot of the stuff in Visi easy to write... basically some of the computation logic (including laziness) is already built into Haskell, so rather than writing a lot of the pieces myself, I get them for free. The cost with Haskell is that pure, lazy, monadic is hard and slow and long-slogging (although it's proved remarkably solid.)

I revisited the Haskell vs. OCaml decision last week because Paul kindly spent a lot of time discussing the strengths of each language/runtime. But at the end of the day, Haskell keeps proving itself the right choice for Visi.

I hope that going through the history of Visi and Visi.Pro and the choices I made in the system and in the tools will help folks understand why Haskell (okay, there's no compelling Haskell vs. OCaml argument that I've made) and more importantly why the choice of platform dictated a language choice other than the language I am most comfortable in: Scala. Put another way, this is the first time since I wrote Mesa in Objective-C and C++ that I was not serially monogamous with a language. ;-)

Permalink

No, I don't owe you scala-tools.org

Sat, 21 Jan 2012 09:45:00 -0800

Apparently I'm a jerk for shutting down scala-tools.org. Apparently, I'm an egomaniac for deciding not to sell the domain for "more than $0" even though nobody has made a legitimate offer for the domain. [Note: James Iry asked the question on Twitter. It was a perfectly reasonable question that I answered as best I could in 140 characters. I answered him and there were subsequent posts from others that personally attacked me for not doing things the way they think I should. Posts from others who attacked me for talking about using scala-tools.org to mourn the losses that I see in Scala-land. This post is *NOT* aimed at James. I like James. I respect James. James represents some of the very best of the Scala community and he was one of the folks who energized me about Scala and gave me hope that Scala could be a "local maximum of research and practical in computer langages." I am deeply sorry that James read this post as something about him.]

I plan to transition the scala-tools.org accounts and such to another provider. These plans will be rolled out on Tuesday. There will be ongoing support for a Nexus/Maven repository for the Scala community. The community will continue to have a place to share components.

What I do not have an interest in is selling or transferring the scala-tools.org domain. There is not a way to set a reasonable price for the domain. The domain has value because of the brand built over the last 3+ years of its existence. How do you price that? How do you price the value that the domain has brought to the Scala community as a whole? Whatever I think that monetary value is, it's likely a few orders of magnitude more than others think the value is. It's not even worth, in my opinion, trying to price it.

So, what value does scala-tools.org have to me if not a monetary value? That's up to me. Sorry, to say, but it's my domain. It's something that I've paid for, worked on, recruited others to work on. Yes, there's a conversation to be had with DavidB and Derek for their work over the years. There's less of a conversation to be had with Josh because, while he helped out, he also dropped to ball on the Nexus skills transfer to Indrajit and Lukas which precipitated my decision to close scala-tools after Lukas expressed extreme frustration with being ignored. I also owe some obligation to Indrajit and Lukas for stepping up when I put out the call for help a few months ago.

To the rest of the community, I owe a reasonable transition to a new hosting solution and that reasonable transition will happen.

But for those of you who have some notion that my contribution to the community over the years creates an obligation to continue to contribute, to keep giving, to keep doing unpaid, community service for you, you're not living in reality. The fact that I have given my time and my effort and my cheer leading and my coding and my writing and my servers and time I could be spending with my family (like the first day of my kids' first spring break that I missed because I had to ward off a DoS attack against scala-tools.org and keep the system running) and all of that stuff does not mean I am forever obligated to keep giving my time, my money, my efforts, and the other stuff that I've given in the past.

My life goal is to leave things better than I found them. I feel a fierce obligation to that who have relied on me, and thus when I make transitions, I try to make them gracefully so that there's plenty of time for others to make those transitions.

When I made the decision in late May to do my next startup, http://visi.pro, in a language other than Scala and on a platform other than the JVM, I made sure to have a graceful transition of leadership in the Lift community and I continue to support Lift and the Lift community because I owe an obligation to those who adopted Lift.

When it became clear that Josh and Derek were not going be able to continue to support scala-tools.org, I put out a call for more volunteers. Given that I was phasing out of the Scala community, I could have just shut scala-tools.org down then. But I asked for help and worked on a transition.

But now, now when it's clear that it's time to shut scala-tools.org down, there's plenty of "you should do this," "you should do that." Well guys, where were you when I put out the call for help a few months ago? What has changed that would make you capable of making scala-tools.org work now?

More importantly, what gives you the right to insult me personally for making a choice about something I own and something I contributed to mightily? If you wonder why I'm sad about the state of the Scala community, just read my blog, watch my Scala Lift Off London keynote.

My way of working through my sadness about opportunities lost, my way of mourning these losses will be expressed on scala-tools.org. I will use something I had a mighty hand in building to express sadness about what could have been.

Will this cause some inconvenience? Maybe. Will this raise awareness about the issues holding Scala adoption back in a way that I've been unable to do in other ways? Maybe. Is this my choice? Yes. Do I owe you more than a smooth transition to another domain and another provider? No. Do I owe you scala-tools.org? No.

Permalink

Sweet, Simple Patterns

Tue, 10 Jan 2012 13:10:42 -0800

I continue to love subletting office space from the Stackmob folks.

Coming into work each morning, I get a fresh bowl of water for Archer and make a pot of green tea for me.

I answer email, checkout on Twitter links, and listen to music. Sometimes a few of the Stackmob folks are in the office. Sometimes, I’m all alone.

The Stackmob engineering group and other tenants wander into the office over the course of the morning and there’s a hum of energy in the geek corner of the office. Sometimes people come over and say “Hi” to me or Archer or both.

Around 11am, Archer reminds me that he needs a walk. We go around the neighborhood which is rich in smells… the kind of smells that are interesting to dogs, but not to humans.

Around noon, I wander out in search of food. The food situation in the neighborhood is not optimal, but there’s generally a good food truck about a block away. Today, it was Nom Nom. I generally bring lunch back to the office and sometimes chat with folks while I’m eating.

The afternoon is a time of focus on work. I’ve generally settled in my mind what I need to get done: some coding, some writing, some thinking… and I work through it.

Around 4:45, it’s time to clean my tea pot, tea cup, water bottle, and Archer’s water bowl. Then it’s time to head back home and settle into the pattern of being part of a family.

I continue to be super-stoked that I’m in the office space that I’m in. The vibe here is sweet and simple and oriented to getting things done. Nobody’s hair is on fire. There are no emergencies. It’s just a place where good work happens and everyone smiles. I love this pattern.

Permalink

Or maybe they're just being quiet

Sat, 31 Dec 2011 17:00:40 -0800

Are there other intelligent species out there? Some people think that we are the most advanced civilization in the galaxy.

I think we're simply being nieve. Think about it. If you're an advanced civilization, do you want to advertise? Do you want to send out radio signals that others can decode?

Or, perhaps, you want to be quiet. Why?

Well, if there are more advanced civilizations out there and they are aggressive, do you want to invite them to come visit you? If the Native American's knew what the Europeans were going to do to them, would the Indians have helped the Mayflower settlers survive?

Now, think about cryptography. The best crypto algorithms make meaningful signals appear like white noise. And we're just 35 years into the likes of RSA cryptography. Imagine the kind of cryptography we'll have in 50 or 100 or 500 years. All our communications will appear as white noise to any listener.

Perhaps every civilization goes through the "hey, we're here... come visit us" phase when they have sub-light speed communications technology. Maybe some of them get welcome visits. Perhaps others get unwelcome visits. But I'm betting by the time civilizations get supra-light speed communications and transportation technology, they stop advertising. They use exclusively encrypted communications technology. Those folks are not going to be visible to us. Yeah, maybe we'll get some civilization's version of I love Lucy in a radio spectrum. But I'll bet by 2050, every form of communications on earth will appear as white noise to an outside observer.

So, in order to get a more realistic probability that we'd observe another intelligent civilization with SETI, we should be estimating that radio signals will only be coming from a civilization for about 150 years and then all the radio signals will appear as white noise. So, even if there are 10M civilizations in our galaxy and those civilizations have existed over 1B years, but they only broadcast non-white-noise signals for 150 years, what's the probability of us hearing one of their signals. That number is pretty low.

I believe there are a fair number of advanced civilizations in our galaxy, but I don't think they are hanging out a "welcome" sign.

Permalink

The whole economic downturn is Groundhog Day

Wed, 21 Dec 2011 11:32:45 -0800

My mom passed away this year and in March, I went back to Providence to clean out her house.

I shipped back some boxes of stuff including a paper I wrote on the Great Depression. I wrote this paper when I was in 8th grade. I wrote this paper when I was 14 years old. I wrote this paper in 1978... 34 years ago. I wrote this paper just 2 years before I started actively supporting Ronald Reagan in his Presidential bid. This paper was not a work of greatness, but simply a regurgitation of settled doctrine written by a Libertarian-leaning conservative teenager.

A couple of process notes. I wrote the report on a manual typewriter. It was written on onion skin paper. It was my graduation thesis from middle school. I did the front cover artwork myself... and quite frankly that artwork could be stamped with #OWS and it would nearly perfectly describe the national and world situation today.

Reading the report gave me the chills. We are experiencing the very same economic and social issues today that we, the United States and the world, did back in 1930s. We are hearing the very same arguments about what to do as we did in the 1930s. I'll go over the few differences that I can identify in the 2007 vs. 1929:

The government stepped in in 2007 and kept the Lehman collapse from totally destroying the economy. Struggling with 15% effective unemployment does not create as much political will as does dealing with 25% effective unemployment, especially when there are social programs to mitigate some of the issues. Because far fewer people are dying of starvation, it's harder of overcome the reluctance to do the simple and correct thing: massive government spending on infrastructure.
Obama came into office earlier in the pain cycle than did FDR and Obama has significantly less spine than FDR. So, Obama has much less strength and much less political will do make things happen the right way.
There was not a farm price collapse during the latest cycle.

Things that are the same:

Increasing income disparity that led to lack of money to consume and ultimately a consumption-led collapse.
Real estate speculation including speculation in Florida.
Demonstrations with 30,000+ people in New York that were broken up by police in riot gear.
The impending European Economic Union collapse.
Quotes like: "We can't squander ourselves into prosperity."

So, in the 1930s, we had never faced an economic problem like the Great Depression. We solved the Great Depression with massive government spending and a post World War II commitment to the middle class which created a huge pile of job creators called well educated consumers.

I had the pleasure of meeting Elizabeth Warren a few weeks ago. Somebody in the audience asked her why the banks, folks with underwater mortgages, and the government couldn't work together to find a solution that was not the current "lose-lose" situation of foreclosures which cost the banks lots of money and drive down real estate values and take away people's homes. Her answer was, "It's like when there's a fumble in football. Everybody piles on the ball and the referee has to pull people off the pile and determine who has the ball. While nobody likes being in the pile, finding out that your team lost the ball is worse, so people stay on the pile for as long as they can."

I think the current economic situation generally fits the same paradigm. Everybody is afraid of making it worse by making the major social and economic changes that will create jobs in the near term, protect and grow the middle class of consumer/job creators, restore a strong infrastructure to the United States which will make every business in the US more competitive, and restore faith in dealing with financial institutions and the government. It's a big mess. It's a nasty scrum. We know how to fix it, but there's so much entrenched fear that the changes will cause short term pain to certain groups (the 1%), that there's not enough political will to make the changes.

So, please read my 34 year old report. Please ask yourself, "did a 14 year old David Pollak presage our current economic situation or are we doing the Groundhog Day thing?" And then take the appropriate action that will set us on the same path that FDR set us on in 1933.

Permalink

The joy of watching skilled professionals

Tue, 20 Dec 2011 11:42:26 -0800

When the whole software craftsmanship thing started a few years ago, I kinda poo-pooed the idea of developers as crafts-people. I rather thought of software engineers as, well, engineers. But over the years, I've come to embrace the idea of craftsmanship as a broader concept... one of caring, diligence, maybe a hint of passion, but most of all of pride in the hard work that leads to a job well done.

I am not overly tolerant of folks who don't work hard or do a good job. Those who have worked for me are chuckling right now, "Not overly tolerant"... bwa hah hah... like "The Arctic is not overly warm." But I do really enjoy seeing people do an excellent job... a job worthy of a craftsman... I love watching skilled professionals do their thing.

Over the past few months, I've been removed from my comfortable high technology world and thrust into the world of construction. Sometime during our month-long trip to Europe this summer, the sewer in our house backed up into the ground-floor in-law unit. We had a house-sitter (thus the sewage), but he did not go into the in-law unit, so the sewage sat there for many weeks. The mitigation firm removed the sewage from the unit along with most of the carpet and some of the dry-wall... leaving us with concrete and studs in the unit. We embarked on a project to restore the unit as well as making it accessible so that my father-in-law could move in.

For the past few months, I've been dealing with contractors and other kinds of folks in the non-high-tech industry. It's been a learning experience... and most of what I have learned so far is that construction and building web applications is no different. But I digress.

One of the "surprises" that our construction folks foisted on us two days before the construction was supposed to begin was that we had to remove all items from the unit rather than their earlier indication that we could use one of the rooms as temporary storage until the work in the garage was complete and then move the items from the temporary storage room into the garage. We need to trench the garage in order to replace the plumbing so that we never have another sewage backup. So, the initial idea of music chairs with the stuff we needed to store vanished in a black, greasy puff of "miscommunication."

So, with about 40 hours, we needed to move about 1,000 cu/ft of stuff from the unit into storage. We needed help. And, being the Internet research godess that she is, @divaesq found us help.

We contacted Andrei Movers, some guys we found in the Internet, to help box, haul, and pack into storage the pile of stuff we needed to move. Andrei and his crew were amazingly skilled professionals. They did an amazing job and it was a joy for me to work with them.

Imagine the grumpiness and grouchiness that hung like a cloud over the 4 tons of stuff we needed to move in 40 hours. Imagine my crankiness at the construction folks who foisted this fire-drill onto me. Yeah, it was non-trivial.

Andrei showed up and his crew had a tremendously positive attitude. "We'll make it happen," he said... and he and his team did. Any boxing the needed to be done, Andrei and his crew did it efficiently and carefully. They loaded a U-Haul truck with tons of stuff in such a well planned way that not a single box moved in the truck on the 7 mile drive to the storage locker. They unloaded the truck into the storage locker with grace and precision. Yes, there was some back-and-forth about the optimal way to stack the boxes so there would be a strong structure. It's the same kind of back-and-forth that I hear from excellent developers all the time.

At the end of the night, Andrei made our insurmountable problem just plain go away. And watching Andrei and interacting with him, I was energized... he made me believe that we could solve the problem and together we did.

I'm writing this post because I always love to see skilled professionals do their thing and watching Andrei and his crew do their thing brought the same kind of joy to me that watching a team of excellent software guys does. And I wanted to share.

I am also writing this post as a way of letting folks in the Bay Area know about an excellent team and as a way to drive business to a team that can get it done and get it done right. So, next time you need some moving or hauling or other stuff like that, please call Andrei Movers at 510-750-9255. And I hope you get the same pleasure at seeing these skilled professionals do their job as I did.

Permalink

Mmm... sweet tasty state monads

Mon, 12 Dec 2011 15:24:06 -0800

One of the really challenging things about Haskell is the lack of state... or more specifically, the lack of global state.

In Scala there are a ton of places you can put state (stuff that's mutated across method invocations.) You can use globals, thread-local variables (something that Lift makes heavy use of), and instance variables. Here are some examples:

Global:

object MyState {

private var cnt = 0

def inc = synchronized {cnt += 1}

def dec = synchronized {cnt -= 1}

def getCnt = synchronized {cnt}

}

Instance variables are pretty much the same, except you have to pass the instance around:

class MyState {

private var cnt = 0

def inc = synchronized {cnt += 1}

def dec = synchronized {cnt -= 1}

def getCnt = synchronized {cnt}

}

In Haskell, there's no mutability, so if you have a reference to a chunk of data, the data will never change. In order to change the state, you must create a new instance, but any holder of the old reference see the old data. In general, this is a GoodThing™ because you don't have to worry about state changing out from under you, but it makes accumulating state pretty difficult. For example, if we want to count the number of hellos and goodbyes and accumulate the names of the people who said Hello or Goodbye, it's easy to do:

val state = new MyState

val greeters = List(("David", "Hello"), ("Archer", "Woof"), ("Annette", "Bye")).flatMap {

case (name, "Hello") => state.inc ; List(name)

case (name, "Bye") => state.dec ; List(name)

case _ => Nil

}

At the end of the above execution, we have a list of people who have made greetings and the side effect of updating the state object with the net Hello/Bye count.

In Haskell, without the State monad, we'd have to accumulate both the count and the names and then fold across them:

list = [("David", "Hello"), ("Archer", "Woof"), ("Annette", "Bye")]

gTest (name, "Hello") = (1, [name])

gTest (name, "Bye") = (-1, [name])

gTest _ = (0, [])

acc = map gTest list

sumIt (i1, n1) (i2, n2) = (i1 + i2, n1 ++ n2)

summed = foldr1 sumIt acc

As the state that we carry around becomes more complex, it. Further, having to fold our result in sumIt is counterintuitive. We've already done the calculation, why do we have to fold the pairs into the final result?

There's a simpler way to carry state around in Haskell. Here's an example:

-- the stateful test

sTest v =

i <- get -- get the current state

case v of

-- update the state and then return the value

(name, "Hello") -> put (i + 1) >> return [name]

(name, "Bye") -> put (i - 1) >> return [name]

_ -> return []

sAcc = (mapM sTest list)

sSummed = mapFst concat $ runState sAcc 0

In the above code, we're separating the state from the core name calculation. The get function get the state into the variable i. Depending on the greeting, we increment, decrement or ignore the state.

Now, the thing that confused me for years about the State monad was where "get" was getting from and where "put" was putting it. Well, it's like this, the State Monad is actually some accumulated functions that are "played" by the runState function. sAcc function returns a bunch of functions. The runState function takes that function and a seed value (in this case, 0), and plays the function to get the resulting information and the resulting state.

The State Monad makes stuff like parsing and keeping state while parsing (e.g., the current count of AST elements we've encountered) is very powerful and makes Haskell code that is in effect imperative a lot easier to write.

I recently tried to implement Hindley-Milner type inferencing without the State Monad and the code was nasty and complex with lots of state being returned all over the place (see https://github.com/visi-lang/visi/commit/cfe7d6fe5a6b25795a1c32ab2f18cfcbad1f9b54#diff-1 and all the atv, atv', etc. state) and then https://github.com/visi-lang/visi/blob/b8f06c73f66596af8333ab09d9528bada068a188/core/src/Visi/Typer.hs Note that all the state stuff is handled in the State Monad rather than cluttering up the logic of the code. I've come to really like the State Monad a lot... it makes me feel somewhat imperative while being functional. ;-)

Permalink

Scala's version fragility make the Enterprise argument near impossible

Wed, 07 Dec 2011 09:53:50 -0800

I have been working with Scala for more than five years. In those five years, I've seen Scala evolve and seen the ecosystem and community evolve.

An attribute of Scala is that the Scala compiler generates fragile byte-code. This means that all the code in an executable (JAR or WAR) must be compiled with the same library and compiler versions. If you're using Scala 2.9.1 in your project, you must compile against a version of Lift that's compiled against Scala 2.9.1 and all the Lift dependencies must also be compiled against that version of Scala. This is a reason that Lift has precious few Scala library dependencies. It's also a reason that Lift is sprawling... there are a lot of modules in Lift that need to be cross-compiled and rather than having an external ecosystem of modules, we need to make them part of Lift to make sure they are all compiled against all the versions of Scala that Lift supports.

The by-product of this issue is that it's nearly impossible to test Lift or a Lift-based app against a pre-release of Scala. This is one of the many reasons that there's almost always a "critical bug fix" release of Scala a few weeks after any major release. The first chance that the broader community gets to use the ecosystem of Scala libraries and frameworks is after a release and after the cascading compilation/test/release

A year+ back, during the 2.8 development cycle, some of us got together on the "Fresh Scala" project. Basically, the idea was to have continuous integration of the major libraries in Scala-land and weekly milestones so that the community and the ecosystem could give better feedback on new versions of Scala. Due to a lack of volunteer time, we never got the CI system built and here we are a year and a half later with the same Scala version fragility issues, but one thing has changed.

Scala is no longer exclusively an academic and community effort. Scala the language and some associated libraries are now part of an enterprise-targeted suite from TypeSafe. TypeSafe is a huge (20+ heads) venture funded company that is supposed to be bringing Scala to the enterprise. But Scala's version fragility creates two huge costs for complex enterprise-type systems:

All dependencies must be compiled against the exact same version of Scala and every referenced Scala library which makes in-house builds when there are multiple teams amazingly complex or time-consuming
External libraries (like Lift) cannot have multiple layers of dependencies because you need to match the exact version of every library and Scala version all the way down

So, if you're in an organization with more than 2 or 3 development teams that are all generating internal Scala code, you can generally agree on the version of Scala and the various libraries that you're going to use. When a new version of Scala or the libraries are released, you can get a couple of people in a room and choose to upgrade (or not.) But once you get get past 3 teams, the coordination efforts become significant. Trying to get 10 different teams to coordinate versions of Scala and Lift and ScalaZ and Rogue and Dispatch, etc. is going to be a significant cost.

As a library builder, the version fragility issue means that our library is not nearly as good as it could be and we can't service our community the way we want.

In terms of servicing the community, we cannot have predictable releases of Lift because we try to time our releases such that Lift will be released soon after a new version of Scala is released. Given that Scala release schedules are not predictable and given that we need to wait for the dependencies (e.g., ScalaCheck and Specs) to be upgraded, we can't answer simple questions like "when will Lift support Scala 2.8.2?"

More importantly, there are few external Lift modules and they are generally not available except for specific versions of Scala and Lift and there are always issues when there's a version mismatch. Modules like Rogue and Reactive Web are available for a small subset of Lift/Scala version combinations. There are dozens of folks who have smaller modules for Lift, but none of them are cross compiled for the last year's worth of Scala and Lift releases. This basically means you can use Rogue (something I'd suggest) or you can use Reactive Web (something else I'd suggest), but you can't use them in the same project because there's no guarantee that they are compiled against the same versions of Lift and Scala, nor is there any guarantee that they will be compiled against the versions of Scala and Lift that you want to use.

So, the Scala library and framework ecosystem is about as big as it can get given the Scala version fragility issue. Because its not possible to have multiple layers of library dependencies in Scala, there will either be monolithic projects like Lift or islands of projects like ScalaZ. And at this point, Scala does not have enough native libraries and frameworks to make it enterprise-worthy. Yes, you can use Java libraries, but as others have found, the cost of wrapping Java libraries in Scala is often not worth the cost/effort.

What have I done about the issue? Over the years, I've raised the issue privately and it has not been addressed. I have tried to organize a project to address part of the issue, but haven't had the volunteer uptake for the project. I have had discussions with Paul Phillips about what could be done and Paul, Jevgeni and I even worked out a system that could be integrated into the Scala compiler to address the issue in 2010 when we were at the JVM Language Summit. The best answer so far is the Migration Manager, a closed source tool that may address some of the issues (I haven't tested it because I'm not going to use any non-OSS software for compiling or testing Lift to avoid any legal issues.)

What we need:

We need something like Scala Fresh so that there's continuous integration across the major (and not-so-major) libraries and frameworks in the Scala ecosystem
Something baked into the Scala compiler that makes the version fragility issue go away

What you can do:

Blog about the issue and raise awareness about the version fragility issue and how it has impacted you
Next time you're on a TypeSafe "what would you pay us money for?" call, please tell them that a vibrant Scala ecosystem is critical to wider adoption and the version fragility issue is significantly impacting both internal development efforts and ecosystem growth. In order for your company to consider buying anything commercial from TypeSafe, you need broader internal adoption of Scala and that comes from reducing the transaction costs of using Scala including eliminating the version fragility issue in the open source parts of Scala.

Why am I writing this post today? The version fragility issue has come up a bunch of times on the Lift list and the Lift committers list in the last week. It's a non-trivial piece of pain and rather than being "go with the flow" about it, I figured I'd write about it and let folks know that it's pain and it's pain that could go away.

Permalink

Loving my new office space

Tue, 06 Dec 2011 13:08:42 -0800

My wife, @divaesq, was right. I needed to get out of my cave.

I’ve always been someone who values coding in a cave, all alone. I spend a substantial amount of time on email, interacting with the Lift committers, many of whom I’ve never met face to face and with the Lift community as a whole.

When I started Visi.Pro, I had a choice... get a cave-like office near the house for a short community and cheap, plentiful space or go South of Market in San Francisco and interact with real people, in real life.

I spent nearly a month looking for the right office space. I wanted something that was a reasonable commute (the Mission, for example, was out.) I wanted something that had a good vibe. Some nice to haves included being dog friendly and hanging with Lift users... I didn’t want to spend half my day “discussing” the merits of Lift vs. xxxx.

I found a space that was way beyond my wildest dreams. I’m subletting from Stackmob.

The Stackmob folks are really cool people doing amazing things in the mobile space. They’ve got a great space that’s dog friendly (Archer likes coming to work with me.) Stackmob is using Lift... so going to lunch with them is fun. They host meetups and other social events in their space. But, perhaps the most important thing is that the space has a vibe of awesomeness.

I don’t know how to quantify the vibe, but it’s one of quiet excellence... inclusive success... that special place where a win for the customer is a win for the employee is a win for the business.

Anyway, I’m stoked to be subletting from Stackmob... and they have some more desks available. So, if you’re doing Lift development or cool Haskell stuff or beautiful iPad apps or just need a space with friendly dogs and amazing geeks, please ping the Stackmob folks and see if you can sublet here, too.

Permalink

Maybe Occupiers just want to participate

Wed, 23 Nov 2011 09:40:11 -0800

I am a firm believer in the Occupy movement. I believe in the awesomely peaceful way the Occupiers are occupying. I believe in the freedom on individuals to speak and march and demonstrate for what they believe in. This applies to the Tea Party folks, the Occupiers, and even the Illinois Nazis. I believe in the Occupiers' General Assembly and wish that there was a way to have a General Assembly in every city and town so that anyone, anytime could take the mic and say what was on their mind... say it to people who have the courage and dignity to listen.

I don't think it's possible to speak for the Occupy movement because the individuals Occupy seems to me to have tremendously divergent ideas, issues, things of importance.

But one thing is clear, that Occupy as a whole embraces participation. And isn't that the very thing that made our country, the United States of America, the Great Melting Pot, the most amazing country ever (sorry to my foreign readers, but I do have a strong streak of patriotism)? We all participate. We are all given the opportunity to participate and we all bring our best to the table and do amazing things.

Cast your mind back 80 years ago... cast your mind back to the Great Depression when we had streets filled with people who needed to take a bath and get a job. 80 years ago, during the Great Depression, just like today, the vast majority of those folks who like to do both. They would like to participate in the dialog and job market in this country... it's just that they had been and have been shut out.

But 10 years later, 70 years ago, those great unwashed masses participated. They participated in the United States' war efforts. Given a chance to participate, those great unwashed, unemployed "whiners" gave everything, gave their lives, gave their toil and hard work to win World War II. Those great unwashed, unemployed masses saved our country, our way of life, and our economic system.

So, maybe we should think about how to structure our society, our business climate, our political system to allow for more participation. Given the opportunity, just like during World War II, everybody in this country, every citizen, every Occupier can and will do their very best to participate and make everything they touch better, everyone they touch happier, and the world a great place.

Let's open our cities and open our ears and open our hearts to the Occupiers. Let's join them in peaceful participation. Let's peacefully discourage our government from beating and hurting the Occupiers. Let's use our words. Let's use our ears. Let's all do our best, together, to make our country and our world excellent.

Permalink

I want a simple PDF caching App (for Christmas)

Wed, 23 Nov 2011 08:32:00 -0800

I want an App... it's simple. If I'm reading something on my iPad on Twitter, in the browser, in Flipboard, I press a button and all the PDF files linked from the page I'm reading get downloaded and made available in iBooks as well as on all the Macs I own (I own quite a few Macs... office, travel, family). The App should also be available on my Mac and automatically move PDFs to my iPad and other Macs.

Yes, this is kinda like Instapaper, but I don't like the way Instapaper re-formats things and it generally doesn't do well with PDFs.

Basically, I'm looking for a library building app that makes my library available (like my Kindle's library) where-ever I am, even if that's on an airplane that has no WiFi.

Oh... and this App should not require me to go to the command line or mess with wacky settings or anything.

Permalink

What I learned from my dog, my kids, and the open source community about being a better person

Tue, 22 Nov 2011 13:51:36 -0800

Participation & contribution, clear expectations & consequences, and respecting energy levels are what it's all about.

8 years ago, I dropped out of the start-up rat race, got a dog, got my wife pregnant, and then embarked on some long range personal growth including starting the Lift Web Framework project.

Archer arrived on December 10th, 2003. Archer is a Collie Dog... a smooth coated collie. Collie dogs are generally smart (Archer is) and very sensitive (Archer is.) They love being members of the pack... part of the team and they do well with positive reinforcement. They groove on having a job, doing their job, and getting praise. This methodology is the philosophy behind Siruis Puppy, where we went to learn to make Archer a happy member of our pack.

I learned how to form commands for Archer that he'd understand and how to catch him doing what I wanted him to do and giving him positive consequences (rewards) for desired behavior. I learned that puppies need to play and learn about each other... that puppies speak dog much better than I do. I learned that letting Archer get in tussles with other dogs was part of his growth and socialization. Archer is a remarkably well socialized dog. He loves to meet new people and dogs. He interacts with happiness and enthusiasm to almost any animal. He's polite to cats and small dogs. He has been able to get almost every dog he's ever met to play with him. He's also good at ignoring the barking, aggressive dogs (something I'm still learning.)

A lot of what I learned from Archer... from watching him at the dog park... I applied to my kids. From the very start, we were positive with the kids. We got on good time-rhythms with the kids so they knew that they could expect food, sleep, etc. at particular times. We worked to wait for desired behavior before giving positive consequences (e.g., paying attention to them when they stopped screaming rather than yelling at them for screaming.) When the kids got into the pulling glasses off faces phase (@divaseq and I both wear glasses), we told them the rules ("Glasses stay on faces") rather than a knee-jerk "No."

Rewards for the kids is mostly family-time and our attention. Punishment, negative consequences, is usually disengagement. If one of the kids is not doing good listening, not behaving in a way that we've asked them to behave, we ask them to sit on the stairs or go to their room. Behaving poorly has the consequence of being separated from the group until the individual can align their behavior with expected family behavior.

Natural consequences is critical. When my son knocked over a full box of K'Nex the other morning, the consequence for him was to pick it up... every single piece. It took a lot of time and he really disliked the task. He's been more careful since.

Letting the kids "skin their knees" is another part of the consequences thing. We let our kids make mistakes, get hurt, and explore their limits. Sometimes I'll point out that what they're doing could have a suboptimal outcome. They generally ignore me. When that suboptimal outcome results in a negative consequence (running too fast and falling down), they get to learn a lesson from experience rather than just from my words. No, we don't let them spark up flame throwers or do wicked dangerous things. But they do use sharp knives to help us cook and they do use the stove. A couple of nicks with the knife and small heat ouchies and they are a lot more careful... and they are becoming quite skilled in the kitchen.

And yes, we try to set out clear expectations. We tell the kids what we want them to do, the way we want them to behave. We do this with words and with our own actions. We say, "Please" and "Thank You" and ask the kids to be polite and do the same. We watch for the kids doing the right thing and thank them for behaving in a polite way.

The dog and the kids have really done well by being part of the pack, the family, the team. But does this work for adult humans?

Yes, yes it does.

The Lift community is remarkable. Lift is remarkable. It's different. It breaks the standard MVC paradigm and it relies on an obscure language, Scala (okay, Scala is less obscure than it was 5 years ago when Lift began its life as Scala with Sails.) Lift's documentation is weak, so for people to want to use it, they must really, really want to use it.

But it turns out that the Lift community and the Lift codebase work the same way as my dog and my kids work.

We value politeness in the Lift community. I've had more than one rant on the subject. Why? Because with politeness as a foundation, we can explore lots of different ideas and concepts. We can include lots of people in the conversation. Sometimes we talk with code and sometimes we talk with English, but the conversation can be broad ranging and can lead to a better result because we've all participated in it.

We can each be the giants whose shoulders we each stand on (there's a turtles all the way down meets MC Escher thing floating around my brain with this one... I'll flesh out the metaphor at some point.) We can share and grow and learn from each other. The value of participation and contribution, being part of the team, the pack, the community, has concrete benefits (more, better code and *ahem* documentation) as well as the non-tangible benefit of making everyone in the community feel better about themselves and about how they spend their time.

Sometimes, we get community members that don't belong in the Lift community (just like Archer will occasionally meet a dog he doesn't want to be around.) Generally, those people self-select and go elsewhere, and the community continues to be a place where people title emails like "Unexpected behavior with anonymous function" rather than "Bug in sucky Lift."

I have learned from the community that care and feeding, setting expectations, rewarding excellent participation with recognition, and spending time with those who do exemplify the community leads to a community of beauty and excellence.

The thing I'm working on now is recognizing the energy levels and energy flows. The kids wear their energy levels on their sleeves. The dog... well, he's a dog and he doesn't really give me a very good read on when he's looking for a walk, a nap, some quiet petting, or some dog-on-dog time. The Lift community is most interesting from an energy perspective. Sometimes, the Lift list is all newbies asking noob questions. Sometimes, it's a bunch of folks working towards a deadline. Sometimes people need new features and sometimes they need better documentation and every once in a while, I get the sense that the collective Lift list needs a beer and the chat.

I am learning about my energy flows and how my energy levels are influenced by those around me and how I influence those around me. I am feeling out how, now that it's official that I'm just a Lift committer, not the BDFL, is impacting the Lift community as a whole and the Lift committers. It's interesting to see how my family reacted to the last month of my "shipping mode" and "code mode" energy levels as I got Visi.Pro launched. It's also very interesting for me to spend more time in the Lift community and feel the community energy now that it's public that I have two children (Lift and Visi.Pro), not just one.

But I've learned so much and grown so much as a person by learning from my dog, my kids, and the Lift community.

I have learned that positive participation... being part of something... is tremendously powerful. I have learned that asking for what you want and saying "Thank you" when you get it gets you more of what you want. I have learned that being part of a family, pack, community motivates dogs and humans to do amazing and beautiful things.

I am so Thankful (this is a Thanksgiving post after all), about how each of these constituencies, family, groups, packs, teams has taught me to be a better person.

Permalink

The Rule of Law & Midnight raids #OWS

Thu, 17 Nov 2011 16:39:48 -0800

Rule of law is important. It’s paramount.

Our country, the United States, was founded on the Rule of Law.

The Rule of Law means that no person is above the law. The law applies to the President of the United States as both Richard Nixon and Bill Clinton found out (the hard way.) This applies to everyone in the country... everyone who sets foot on the US soil (except diplomats) is subject to the same Rule of Law, the same Due Process as everyone else.

It really made me angry the way the NYC police evicted Occupy Wall Street from Liberty Square. This is not an argument about the substance of the eviction. This is an argument about the process.

Now, it’s supposed to be in the United States of America that there are a bunch of laws that are published (either as statutes or as common law decisions made by courts.) While there are exceptions to this rule (which is not the subject of this rant), in general, we all know what the rules are.

Now, sometimes there are disagreements about what the rules are. And when there are disagreements about what the rules are, people go to court and, in the light of day, in front of the entire public, parties argue about various sides of the issue. Sometimes, the issues are simply matters of law (both sides can agree of the facts) and a judge can make a decision. Sometimes there are facts to be found, in which case, the issue is presented to a jury, the jury finds the facts and then based on the facts that the jury finds, the judge applies the law and issues a ruling.

After a judge issues a ruling, then, and only then, may the police act.

Let’s contrast this with what happened at Liberty Square.

At Liberty Square, the Occupy Wall Street demonstrators have been sleeping in tents and occupying the area. The OWS folks have been engaging in what they call political speech and that speech is the very essence of what the First Amendment protects.

So, we’ve got some facts that need to be found. We need to figure out if the time, place, and manner rules that govern camping in Liberty Square are in fact neutral or as a matter of fact, is the act of hundreds of people pitching tents an act of political speech. This is a fact to be determined. This is not a matter of law, this is a matter of fact.

Why does it matter?

Well, if we’re a government of laws and we believe in the Rule of Law, then New York City would have followed reasonable procedures before evicting the OWS folks from Liberty Square.

Given that the OWS folks have been camping in Liberty Square for many weeks, there’s no argument about exigent circumstances requiring the removal of the OWS folks from the park on a particular date. Given that the OWS movement is represented by lawyers, it would have been both trivial and correct for the city of New York to sue the OWS movement to remove them from Liberty Square.

The city of New York should have demonstrated, by example, that it follows the rules, the Rule of Law, and respects every citizens’ rights. The city should have filed suit to evict the OWS demonstrators from Liberty Square. The city should have followed reasonable legal procedures.

The discourse about wether the OWS movement’s encampment in Liberty Square was political speech or not should have taken place in public, in court, and, in my opinion, in front of a jury that could have found facts.

If the city received a ruling from the court that upheld the city’s right to evict the OWS demonstrators from Liberty Square, then, and only then, should the city have engaged in eviction. That eviction process should be done in the light of day, rather than as a midnight raid. The eviction process should be done under the scrutiny of the press. The eviction process should be done via patience rather through the use of force.

The OWS movement is reminding me about what is means to be an American. We use our words. We share our perspectives. The General Assembly is what democracy, Democracy, is about. We discuss and debate and patiently work through things until we get a result that generally works for all involved. The OWS demonstrators are a reminder that patience and words and unity is what the United States is all about.

Trying to break up OWS with a lawless midnight raid where the press is excluded simply underscores what OWS is protesting against. The United States and New York City are supposed to be governed by laws that apply to everyone and are applied with Due Process. When Bloomberg ordered the police to act in a lawless manner, he showed that he is no more than the petty dictators and despots that our nation has sent our army into Iraq and other places to depose.

So, I am very, very angry about what Bloomberg did in New York. He did not follow the Rule of Law. He did not respect what makes the United States awesome and beautiful. The cost of following process and working in the light of day is low and the cost of midnight raids is huge.

Permalink

It was 20 years ago today...

Thu, 17 Nov 2011 10:06:00 -0800

It was 20 years ago today that I embarked on an epic computer journey that defined my life as a coder and a business-person. On November 17th, 1991, I laid down the first lines of code for the Mesa spreadsheet.

Mesa was a spreadsheet for NextStep and continues (without my help) to be a spreadsheet for OS X (which is the updated version of NextStep.) At the time I started coding Mesa, the spreadsheet category on NextStep was surprisingly crowded with an entry from Lotus, the amazing, but non-traditional Improv, and WingZ from Informix. Additionally, a venture funded company was working on traditional, native NextStep spreadsheet called PowerStep.

I built a company, Athena Design, while building Mesa. We launched Mesa in August 1992 and within 6 months, Mesa was the dominant spreadsheet in the NextStep market. While some called me a Lone Wolf, in fact Athena and Mesa were an amazing collaboration by a stellar team. Put another way, I built Mesa and Athena with a little help from my friends (at this point, please start listening to the Beatles Sgt. Pepper’s)

What was amazing about Mesa was we built it based on talking to spreadsheet users in the NextStep community. NextStep sold heavily into financial services and those folks wanted to be able to tick real time data from Reuters and other data feed services, perform calculations, and trigger external events. Mesa did this. In fact, working with IBM and Reuters in the late 1990s, I discovered that Mesa was the original art for the “Real Time Spreadsheet.” Additionally, Mesa was an early example of a spreadsheet component that could be embedded in other applications (maybe it was the first, although my friend Debby Meredith apparently wrote a Smalltalk spreadsheet that could be embedded as well.)

While the Next, and subsequent OS/2, OpenDoc, Taligent, and Be markets were not large enough to support a growing company, what Mesa represented was that a small team of amazing people who listened to users and mixed in some of their own vision could fundamentally change the landscape.

Five years ago, when I discovered Scala and started coding Lift, I made the decision, once again to eschew the status quo of MVC and, instead, learn from what users and developers need. For the last 4 years, Lift has been the best web platform for building Comet, Ajax, and Secure web apps. Lift has grown and evolved, yet the core decisions about how Lift worked have remained correct and solid, and users like Foursquare and the Guardian agree.

Modeling Lift and Mesa on user-needs rather than machine needs is a different approach. It’s an approach that centers the product on the user rather than the machine. Sometimes the difference is subtle and sometimes it’s not, but in the end, and as Kernighan and Ritchie taught us, people time will always get more valuable and computing cycles will always get cheaper.

Now, today, 20 years to the day after I started my epic journey, I am launching into a new journey: Visi.Pro.

Visi.Pro brings Cloud Computing to the rest of us. Visi.Pro is HyperCard for the iPad. Visi.Pro lets users build iPad/iPhone apps on the iPad (or Mac) and deploy them seamlessly across the Cloud including using Cloud resources such as persistence, data sources and data sinks.

Visi.Pro provides a complete environment from language to IDE to run-time to data/app/component emporium. Visi.Pro provides a complete environment for the rest of us to create interactive, powerful apps. A video of the vision, a vision statement, and more can be found at http://visi.pro.

Building Lift and the Lift community, I have realized that standing on the shoulders of giants is the best way to build something world-changing. The best giants around as the folks who make up the open source world and a substantial portion of Visi.Pro including the language and much of the iPad code will be open source under an MPL license. This means that if others want to build stuff off the Visi language, a language so simple and powerful that any Excel or PHP developer will be at home with it, they are welcome and encouraged to do so. You can check out more about the language and such at http://visi.io Please check out the code and the community... please contribute back to the conversation and the code.

Building Visi.Pro and Visi.io will be a long, tough learning process. It’s going to be interesting to learn from the PHP and Excel users in the world. It’s going to be great understanding the kinds of things business people and moms and teachers want to build on the iPad. It’s going to be like skiing double-diamond black slopes to build an invisible type system into Visi so that the users are always guided to do the right thing. It’s going to be spectacular to get the Program Language Theorists to work through a type algebra (yes, Visi will allow an algebra in the type system) along side a teacher describing a grading app. But what will really generate a tsunami of joy is when people are able to build amazing, interactive, real-time, cloud-backed apps on their iPad. I can’t wait for that day and I know it’s going to be a long, tough, and spectacularly rewarding journey for all involved!

Before I ask you to join me on this journey... before I ask you to join the Visi conversation, I want to talk about some changes in me and how this will impact the Lift community.

For most of my career, I have aligned myself with the dark horses and the lost kittens. I bet on an ousted Steve Jobs and his Next platform. I bet on OS/2 and OpenDoc when Windows was a better choice. I bet on Jean-Louis Gassée and BeOS because he told great stories about RepoDepo. I bet on Java and WebLogic in 1996. I bet on Scala. Some of those amazingly risky bets worked out and others, well, didn’t. This time, I’m betting on the clear winner. It’s clear that iOS will dominate the OS/computing landscape for the next generation. It’s clear that the cloud and interconnected and partially connected computing will be the way it’s done for the rest of my professional life.

So, rather than choosing the dark horse, I’m going to bet on the winner. And rather than choosing an existing category and simply building the best technology for that category, I’m going to create a new category (or more properly revive a lost category... the category of “everyone can program this machine and make it beautiful” category.) I’m going to fundamentally change the way people program for the iPad and the Cloud. I’m going to kick a dent in the universe.

My commitment to Lift remains. I love Lift. I love the Lift community. I remain in awe of the amazing committers. I am stunned and amazed by the folks who have committed to using Lift and I remain committed to Lift and the folks who have adopted Lift. For most of 2011, I have spent about 12 hours a week in the Lift community... working on the mailing list and closing tickets. Most of this year, I’ve been doing Lift-related consulting and much of my contributions to the Lift code-base has been for my clients. I will continue to spend 12 hours average a week in the Lift community. What is going to change is that the tickets I close, the code I write for Lift, is going to be entirely community-focused. It’s going to be based on what the community needs rather than what my clients need. Longer term, I will find a way to make Visi and Lift play together. They are both my children and I love them... although sometimes newborns need a little more attention than 5 year-olds. The key change is that I will not be available to consult on Lift-related projects in the future.

So... what do I want from you?

I want your brains. I want your input. I want your language ideas. I want your app ideas.

Are you involved with Rust, Roy, Haskell or other amazing languages? Have you built stellar iPad apps like Codea and Blueprint? Please join the conversation. We can share ideas and together be more than the sum of our individual parts.

Are you someone who wants to build something on your iPad? Something beautiful and amazing? Something that you can share via the cloud with others in real-time? Something that sucks data from SalesForce, Twitter or a million other data sources or sinks on the Internet? Please join the conversation.

Visi.Pro is going to be a totally epic journey and I look forward to you joining the team and making Visi.Pro life-changing for millions of iPad users.

Thanks!

PS -- Visi.Pro and Visi will always be defined, developed and built with strict adherence to Apple’s policies about what it allows on the App Store and on the iOS. Visi.Pro, like the amazing Codea or Numbers, is a development environment and interpreted runtime. I welcome any public or private (at Apple’s option) conversation with Apple regarding Visi.Pro to clarify the iOS and App Store rules and to insure that what we do with Visi.Pro is done clearly on the correct side of the line and with Apple’s happy blessing because, at the end of the day, our goal is to empower iPad and iOS users to rock their respective worlds!

Permalink

#ows #olsx A day in London

Sat, 15 Oct 2011 07:23:35 -0700

I just spent about 2 hours at Occupy LSX (London Stock Exchange). It was a very interesting and most excellent experience.

I arrived early. There were a few hundred people when I showed up at St. Paul's Cathedral (conveniently located less than a mile from where I've been staying for the Scala Lift Off London 2011.)

The Socialists/Marxists where there handing out information and signs.

But more people started arriving. It was a diverse bunch... from a nice 70+ year old woman who was there to protest using drones to "assassinate people overseas," to a drunk guy who Fox New instantly gravitated to, to lots of you folks with iPhones and other non-trivially expensive gadgetry, to a bunch of folks like me: well dressed 40-something to 50-something.

There was a lot of energy in the crowd and most of it was oriented towards positive and peaceful change. It was clear that everybody there had a vibe that something was broken.

The strongest vibe I got was from people who wanted jobs. That set off a lightbulb in my head. Listening to the criticism of #OWS in the US, there's a lot of talk about "whiners" and "people who are too lazy to have jobs." At least in London, that couldn't be farther from the vibe. The folks here wanted jobs. The folks here wanted to participate in the economy. The sense and extrapolation that I took from the demonstrators was that they have been denied the opportunity to participate in the economy and that's the fundamental thing that needed fixing.

It's a very interesting concept that these folks were in fact demonstrating for a market that they could freely participate in. They want to be part of the solution, to be part of a society that is vibrant and productive. The frustration that I extrapolated from the folks and the handouts and the brief chats I had with people was that there's a walled garden and the 1% are in the walled garden and they dismiss the value of the 99%. But being in the crowd, it was clear that there was a long of value and positive energy that could and in a better world would find a way to make everybody's life a little better.

Speaking of the police, there were hundreds of police there. They were very well behaved. Very polite. There were police on foot. Police on horses. Police in helicopters. Police with still cameras. Police with video cameras. Lots and lots of police.

There were also lots of reporters. Not just Fox News, but reporters from many different countries.

When the demonstrators started to move towards the London Stock Exchange (a few hundred feet away from St. Paul's), I moved with them. There was a line of police blocking all the entrances to the square that the LSX is housed on.

The police stood shoulder to shoulder with a line behind them and mounted police behind them. There were a lot of chants "Who's streets?" "Our streets!" I joined in on those chants, but felt a little weird about it given that I'm not a citizen and they're really not my streets... but I digress.

The protesters continued along the St. Paul's Church Yard alley over to a street near the St. Paul's tube station. The police at this entrance seemed to be very, very nervous. The police were standing stock still, mostly with their eyes on the ground. The mounted police behind them were having significant issues controlling their horses. The mounted police had to finally move their horses to another location. At the same time, the crowd was very relaxed. People were starting to have conversations... almost as if we were at a giant party.

For about 30 minutes, people just milled around, leaving a big gap in between the crowd and the police. Then, some folks came along with a "we are the 99%" banner and markers, put the banner in front of the police and invited people to "draw pictures of themselves." I autographed it with "@dpp" and let others draw or write what they wanted. It was an interesting way to close the gap between the demonstrators and the police.

At some point, somebody started heckling the police. I felt that it was uncalled for. I was there to show my belief that there need to be changes in the way our businesses and government are structured (not radical changes... just tunings). I was not there to get anyone excited, but having thousands of people in one place generates an energy and having that energy shared with others in other cities is a peaceful way of beginning the conversation which can lead to change. When things calmed down, I looked at a few of the police near me and said, "Thanks for being peaceful." They nodded.

That's when things started getting weird. The police formed a line in between the St. Paul's Tube station and the demonstrators. In fact, the police formed a line to stop the demonstrators from exiting as well as from entering the square where the LSX is. That was a weird feeling for me. As a foreigner, I did not want to be arrested (in the US, I'll be a peaceful demonstrator and gladly take advantage of my 1st Amendment rights... and if I'm arrested, so be it... but not on foreign soil.) The police at each of the lines boxing the demonstrators in were manifestly unhelpful. A number of well dress and quite polite people asked them how to get out. The police replied, "I only know that I'm supposed to be here. I do not know about other means of entrance or exit."

I quietly found a way out (back along the St. Paul's Church Yard) and then along some back streets and out of the demonstration zone. In the back streets, there were lots of police riot wagons and lots of police getting into riot gear. This seemed unnecessary as the demonstrators were mostly just hanging out as if they were at a giant party and the few that were being non-chill were simply using their words. Each of the police groups that I passed in the side streets, I made sure to make eye contact and say, "Thank you for being peaceful."

I left the demonstration, but plan to spend time at #OWS in San Francisco. Why, one might ask, would I care about #OWS. I'm well off and generally don't have to worry about where the meals or shelter for the rest of my life are coming from... well...

My success is based on a great confluence of excellent opportunities through my life. Yes, I'm smart and hard working, but that's not enough. I had the opportunity to borrow books from the library and read. I had the opportunity to go to excellent public schools that had reasonable funding and excellent teachers. I got contracts from private sector firms and government agencies (and oddly, the government agencies were the future-thinking risk takers... both FEMA and the RI State Library Services gave me huge contracts to do Apple ][ work even before I could drive a car.) I was able to build businesses because I could hire well trained employees, provide reasonable health care for those employees, and do business in a predictable environment. But the business environment today is much worse than it was 20-30 years ago.

Well trained employees who are foreign born are shipped right out of the country. Health care costs are insane. Deregulated banks are a slime-pit of fees and other junk that sucks hundreds to thousands of dollars out of my business every month. I don't care nearly as much about "government red tape" as I do about private sector red tape. Having more government regulation of large private sector actors makes my life easier as a business builder because the private sector is an order of magnitude more difficult to navigate than the government.

But most importantly, I want a fat middle class. A fat middle class that has enough money to eat well, educate their kids, and spend discretionary income on stuff that I sell or that my customers sell. If they don't have discretionary income, my business opportunities are smaller. If they don't educate their kids, that cost falls to me as an employer. Oh, and if I have to write the health care checks, my business is officially toast. Having a single payer, the government, means that I can focus on my business without having to focus on provider choices, policy management, etc.

The difference to my short term goodness of having 5% less income is minimal. Yet, that 5% means a world of difference for hiring more police and people who build roads and teachers, etc. I'm not going to spend that 5%. Personally, I generally buy government backed securities because I want a safe investment. So, I believe in #OWS for selfish reasons. I believe that having a set of laws that lead to greater transparency in transactions (financial, medical, business, etc.), incentives to create wealth rather than move wealth in a shell game (e.g., high frequency trading) will make everybody better off. I think that giving workers, the middle class, even bargaining power with the oligopolistic corporate entities will make wealth allocation work better over the long run. I also believe that financial transparency (like the SEC) combined with financial entity isolation (like Glass-Steagall) makes for better longer term financial markets because the markets are spending less of each transaction hedging against unknowns. I just never want to have the Chairman of the Fed not get an answer to the question, "How much is currency invested in derivative instruments?" Sheesh... come on guys... we know the precise market value of every public company... why shouldn't we know the market value of the derivatives market?

But I digress... #OWS London was a very interesting experience and I look forward to the #OWS movement growing and peacefully changing the relationships among all of us so that everybody can contribute to society and share in the bounty of what we are all creating.

Permalink

Rounding out the Scala Lift Off London

Tue, 04 Oct 2011 10:32:00 -0700

There are lots of exciting speakers lined up for the Scala Lift Off London 2011.

Additional speakers include:

Brendan McAdams from 10Gen makers of MongoDB. Bendan is the author of Casbah and an all-around excellent Scala dude and presenter. Bendan will be presenting on MongoDB in the Scala ecosystem. Having seen Brendan present in the past, I'm sure the presentation will be informative, funny, and generally rockin'.
Chris Marshall, a stellar speaker, will present ScalaZ in a totally comprehensible way. You'll see how the awesomeness of Scala can be transformed into real-world success in financial services.
Dave Gurnell from Untyped will be presenting real world, high performance, secure REST using Scala and pattern matching.

Tickets and local hotel rooms are going fast... get your ticket today.

Permalink