The Fine Line Between Scope Creep and Proactivity In Software Development

Lately I’ve spent a lot of time internally debating one of my personal software design philosophies. Namely, to what extent, or if at all, should one go above and beyond the idea of MVP(Minimum Viable Product) when developing software?

That is to say, is doing the bare minimum to achieve the desired functionality perhaps even with disregard to the systems around you the best practice? Does the old saying, “If it ain’t broke, don’t fix it?” still hold water? Or is it worth taking that extra 10% of time to slightly re-architect the systems around you with consideration to that extra piece of functionality you’ve tacked on so that it doesn’t explode from the seams the next time you need to add “just one more piece of functionality”?

I’ll be the first to admit that as a fairly inexperienced developer, I have been known to be enthusiastically heavy handed when developing new systems or features. Where a chisel would have been the best tool for the job, more than once my tendency has been to immediately equip the refactor bazooka and fire at will. While I’ve built some pretty cool things as a result and made some pretty impactful changes, the void of code being released in the middle is disheartening. Not to mention the, “Hold your breath… here we deploy.” moments when the code is actually ready to ship.

As part of cleaning up my act and pursuing better development habits I’ve been making a conscious effort to refrain from these sorts of tasks and in many cases push myself in the opposite direction. Which explains how I arrived at this quandary in the first place.

In many cases it pains me when I know with just a bit of extra work, the “just get things done” changes could be made into elegant changes which improve readability, testability, and extensibility in the future. But is that really necessary or is it the case that doing this like this is simply over-architecting. Maybe it is true that if it isn’t breaking the current system or that in which you are trying to implement, then it should be left alone until it does.

My gut tells me that this isn’t the case. If you’ve been working with a code base for an extended period of time and you understand your current feature set and the foreseeable roadmap, then isn’t the extra 10% overhead now worth saving you the headache later on when the system you’ve been hacking things onto actually does explode?

I understand the arguments against this idea, or, at least trying to silo your changes to one scope of work at any given time. But from my experience, as working in a startup, “I’ll make that change in a later pull request”, really means, “It’s not going to get done.” So, with all that being said I ask, where is the line in the sand that one must draw when making these decisions?

Anyway, I’d love to hear from you, what your experiences are on this, and what I’m missing. I wrote this in about 30 minutes, probably in procrastination in studying for my American Politics Final tomorrow so hopefully it’s not too raw.

Golang Password Encryption For Apps and Websites

As one of the first exercises I’ve conducted in Golang as part of getting used to the core language/framework I’ve implemented a simple password authentication package. It uses Bcrypt for actually hashing the password with 64 bit salting created via Dev/Rand which is just standard practice really. This was more an exercise in building something practical and getting a handle on the syntax of doing some slightly more advanced things.

One thing I’ve noticed with Go, especially using this handy dandy Go Plugin, is that it’s forced me to write really clean code(at least I think so). I’ve really been trying to emphasize this lately, but Go seems to really take it to the next level, at least so far with this really simple bit of code. Anyway, I wrote the whole thing to be tested so each function has a well defined responsibility that is easy to call and check.

	package authentication
	// This will handle all aspects of authenticating users in our system
	// For password managing/salting I used:
	// http://austingwalters.com/building-a-web-server-in-go-salting-passwords/
	import (
	"code.google.com/p/go.crypto/bcrypt"
	"crypto/rand"
	"log"
	"strings"
	)
	const (
	SaltLength = 64
	// On a scale of 3 – 31, how intense Bcrypt should be
	EncryptCost = 14
	)
	// This is returned when a new hash + salt combo is generated
	type Password struct {
	hash string
	salt string
	}
	// this handles taking a raw user password and making in into something safe for
	// storing in our DB
	func hashPassword(salted_pass string) string {
	hashed_pass, err := bcrypt.GenerateFromPassword([]byte(salted_pass), EncryptCost)
	if err != nil {
	log.Fatal(err)
	}
	return string(hashed_pass)
	}
	// Handles merging together the salt and the password
	func combine(salt string, raw_pass string) string {
	// concat salt + password
	pieces := []string{salt, raw_pass}
	salted_password := strings.Join(pieces, "")
	return salted_password
	}
	// Generates a random salt using DevNull
	func generateSalt() string {
	// Read in data
	data := make([]byte, SaltLength)
	_, err := rand.Read(data)
	if err != nil {
	log.Fatal(err)
	}
	// Convert to a string
	salt := string(data[:])
	return salt
	}
	// Handles create a new hash/salt combo from a raw password as inputted
	// by the user
	func CreatePassword(raw_pass string) *Password {
	password := new(Password)
	password.salt = generateSalt()
	salted_pass := combine(password.salt, raw_pass)
	password.hash = hashPassword(salted_pass)
	return password
	}
	// Checks whether or not the correct password has been provided
	func PasswordMatch(guess string, password *Password) bool {
	salted_guess := combine(password.salt, guess)
	// compare to the real deal
	if bcrypt.CompareHashAndPassword([]byte(password.hash), []byte(salted_guess)) != nil {
	return false
	}
	return true
	}

view raw

authenticator.go

hosted with ❤ by GitHub

and the corresponding GoConvey tests:

	package authentication
	import (
	"code.google.com/p/go.crypto/bcrypt"
	. "github.com/smartystreets/goconvey/convey"
	"log"
	"strings"
	"testing"
	)
	func TestSpec(t *testing.T) {
	Convey("Authentication Testing", t, func() {
	Convey("generateSalt()", func() {
	salt := generateSalt()
	So(salt, ShouldNotBeBlank)
	So(len(salt), ShouldEqual, SaltLength)
	})
	Convey("combine()", func() {
	salt := generateSalt()
	password := "boomchuckalucka"
	expectedLength := len(salt) + len(password)
	combo := combine(salt, password)
	So(combo, ShouldNotBeBlank)
	So(len(combo), ShouldEqual, expectedLength)
	So(strings.HasPrefix(combo, salt), ShouldBeTrue)
	})
	Convey("hashPassword()", func() {
	combo := combine(generateSalt(), "hershmahgersh")
	hash := hashPassword(combo)
	So(hash, ShouldNotBeBlank)
	cost, err := bcrypt.Cost([]byte(hash))
	if err != nil {
	log.Print(err)
	}
	So(cost, ShouldEqual, EncryptCost)
	})
	Convey("CreatePassword()", func() {
	passString := "mmmPassword1"
	password := CreatePassword(passString)
	pass_struct := new(Password)
	So(password, ShouldHaveSameTypeAs, pass_struct)
	So(password.hash, ShouldNotBeBlank)
	So(password.salt, ShouldNotBeBlank)
	So(len(password.salt), ShouldEqual, SaltLength)
	})
	Convey("comparePassword", func() {
	password := "megaman49"
	passwordMeta := CreatePassword(password)
	So(PasswordMatch(password, passwordMeta), ShouldBeTrue)
	So(PasswordMatch("lolfail", passwordMeta), ShouldBeFalse)
	So(PasswordMatch("Megaman49", passwordMeta), ShouldBeFalse)
	})
	})
	}

view raw

authenticator_test.go

hosted with ❤ by GitHub

Boom, a tested, working password authentication package! Not bad for an hour’s work.

As always, this is my first crack based on what I know, and what I read out there on the interwebs today about Golang best practices. Let me know if you see anything blatantly wrong here and I will make it not so blatantly wrong in case any poor fool uses my code.

Harnessing Third Party Services in Tech Startups

During my tenure as an Industrial Internship Program student I learned many valuable skills and practices as an employee with Mover, a cloud storage related technology start-up based in Edmonton. Perhaps the most perceptible lesson I took away from my employment in the startup industry pertains specifically to leveraging other cloud based software services and solutions to help meet the rapidly increasing and scalable needs of a company that needs to grow as quickly as possible. CircleCi, Amazon EC2, Heroku, UserVoice, and PagerDuty are just a few examples of services that are essential to the operation, analysis, and development of the technology for migrating data between cloud services that Mover provides.

As a startup in any sector, it is important to be as effective as possible with the distribution and expenditure of your internal resources. During my time with Mover, one of the major processes through which we accomplished this was to integrate paid services to support our core technology wherever possible. There are a number of different reasons why we elected to take this route instead of building our own in-house solutions.

To begin with, it is often much cheaper to leverage even seemingly expensive outside technology than spending development resources on it. As a rule of thumb, there is a large correlation between the level of difficulty in creating a service and the price tag attached to licensing or purchasing it. The cost associated with the time it would take to design, implement, deploy, and support these features or tools can often be an order of magnitude greater than simply purchasing the tool or service in the first place. More importantly, you are not only purchasing an easy to implement solution, but also the expertise of its creators. Similar to how Mover has become experts in moving data in the cloud as a result of the time we have spent working with and optimizing our own technology, it makes sense that the same argument applies to other services that you would harness and benefit greatly from.

Another often overlooked benefit is the actual customer support that comes with purchasing or licensing these tools. When an unexpected problem occurs, it is much easier to offload your issues onto the company who created the service, who are again experts at what they do, then to deal with problem yourself. More often than not, these companies are extremely helpful and eager to solve your issues as they are themselves smaller companies who are trying to build their own customer base and reputation.

Cost and convenience aside, the most important reason for integrating and integrating outside tools wherever possible, is that allows you to spend less time working on problems that have already been solved, and more time building your core product. As a startup, you need to move as fast as possible. Any time that is not spent working on your core service has a compounding cost down the line and can become the difference between success and failure in the industry.

It’s nearly impossible to say how much time Mover continually saves with the services we have integrated. But what is extremely clear is just how vital the inclusion of many of these services has been in our technology stack, and will continue to be in enabling Mover to grow rapidly. As a result, I believe this is one of the most important philosophies I have learned and will continue to lean upon in my future as a software developer.

Heroku, Express & Node Bug – 500 TypeError: Object # has no method ‘randomBytes’

I decided I needed to write a post about this error, because the solution took me forever to find, so hopefully some poor soul stumbles across this and doesn’t waste as much time on this as I had to.

Basically if you get a nasty error output from your app when it is live on heroku, the first step is to type “heroku logs” into your terminal. This command pulls your app’s HTTP logs and spits it on screen.

Here was what I recieved:

Error: Can’t use mutable header APIs after sent.

[web.1]: at app/node_modules/express/node_modules/connect/lib/middleware/errorHandler.js:64:17
[web.1]: at next (/app/node_modules/express/node_modules/connect/lib/proto.js:199:15)
[web.1]: at ServerResponse.<anonymous> (/app/node_modules/express/node_modules/connect/lib/middleware/logger.js:149:20)
[web.1]: at generate (/app/node_modules/express/node_modules/connect/lib/middleware/session.js:288:13)
[web.1]: at [object Object].<anonymous> (fs.js:107:5)
[web.1]: at wrapper (fs.js:245:17)
[web.1]: TypeError: Object #<Object> has no method ‘randomBytes’
[web.1]: at ServerResponse.getHeader (http.js:543:11)
[web.1]: at [object Object].emit (events.js:61:17)
[web.1]: at Object.session [as handle] (/app/node_modules/express/node_modules/connect/lib/middleware/session.js:297:7)
[web.1]: at next (/app/node_modules/express/node_modules/connect/lib/proto.js:199:15)
[web.1]: throw new Error(“Can’t use mutable header APIs after sent.”);
heroku[web.1]: Process exited with status 1
[web.1]: at Object.methodOverride [as handle] (/app/node_modules/express/node_modules/connect/lib/middleware/methodOverride.js:37:5)
app[web.1]: ^
app[web.1]: at ServerResponse.<anonymous> (/app/node_modules/express/node_modules/connect/lib/middleware/session.js:277:36)
[web.1]: at Object.uid (/app/node_modules/express/node_modules/connect/lib/utils.js:117:17)
[web.1]: at Object.cookieParser [as handle] (/app/node_modules/express/node_modules/connect/lib/middleware/cookieParser.js:60:5)

While this isn’t the most useful error output, at least to me, some extensive google searching told me that this happens as a result of incompatible Framework versions. Essentially my version of Express was attempting to call methods that don’t exist on the default version of Node.js that Heroku provides.

The Fix! You can read here. Basically you need to specify which version of Node.js you want Heroku to use with your app. I have no porting issues at this point, so I’ve opted for Node.js 0.8.x.

Hope that helps someone, happy devving!

Increase SEO by Linking Your G+ Account To Your Web Pages

One of the first tasks I was responsible for at Mover is linking our blog post articles to our Google Plus profiles. By doing this, Google can then index our search results using verified authors giving our posts greater credibility. Furthermore, the micro-profile of the author will show up under any indexed search results that they are the other of. This feature was released in the middle of 2012.

This is a very simple process:

1) Link to your G+ profile on your web page, and include a ref=”author” tag:

<a href=”[URL TO G+ PROFILE]” ref=”author”>Your Name</a>

Adding ref=”author” informs Google’s web crawlers that they should include the author in search results.

2) Edit your Google Plus Account

To complete the authentication loop, Google will examine your profile to see if you are indeed a contributor, preventing you or others from falsely linking authors to posts or articles. You can update your “contributions” by clicking this link.

Finally, verify that Google’s article profiler can index your newly added author information properly be entering the URL of your site into this tool: http://www.google.com/webmasters/tools/richsnippets

Web Development Fun

My first programming related post! It’s sad that I’ve been neglecting this as it is what I plan on doing for the rest of my life. Maybe it’s a good thing as I’m more wrapped up in training and cooking then my future employment. I will say however that I really enjoy programming and that there are really interesting ways to apply yourself that can be more fun than “work”.

One of the main deficiencies in my education at the University of Alberta in Computing Science is that we spend virtually no time doing web design until 4th year and even then it is really only one class. We do courses on SQL, Java, C, all of which are valuable, but we haven’t actually built any useful products or projects that we can use as examples in our portfolios. Thankfully one of the courses I am currently enrolled in is based around Software Engineering and Android, so at least I will have some experience with Mobile Development through school! Being the smart student that I am, I realized that if I want to find a job when I graduate, I will have to accumulate some useful experience on my own.

One really cool event that I do yearly to build projects is Startup Weekend. It is a 54 hour hack session where basically ideas are pitched and teams are formed Friday night, Saturday and Sunday until 5 is all about building a working prototype, and Sunday evening is about demoing what you’ve got. So far I’ve helped to build 3 website back ends. One for Crowd Sourcing, a Point of Sales system and most recently we tried building a Dating Website which ended up like a really poorly functioning version of Facebook. I personally love this startup environment and hope to one day do this as my full time profession. Having your database schema drawn on half a whiteboard, your site layout on the other, brainstorming ideas on the fly, and building fast makes for really exciting and challenging work days. Personally, I find I learn more practical stuff in one of these weekends then I do in a course over a full semester.

Any ways, I’ve had building my personal portfolio site on my //todo agenda for a long time so I am planning on having that up and running by the end of the Winter semester. It is a good opportunity to learn some new skills, show off my talent as a web designer, and build a kick ass portfolio site for potential employers all in one shot. I am fairly comfortable with html and doing back end work in PHP but my websites always end up looking so “Web 1.0”  like this:

When what I have in my head is something like this:

So as part of my mission to become a better Web Developer, I plan on learning HTML5, CSS3.0, JavaScript, and some Ruby on Rails the right way. I’ve also ordered a book that helps explain how to choose colours, the science behind logical web layouts, fonts, and all of that other stuff that I’m oblivious too as a “hacker” and not a “designer”. The biggest problem I have when user interfaces are required is choosing colors. I swear I tinker with them for hours and can never good combination. To my enjoyment however I found this great site which has been a god send for at least getting sample colours in my projects.

Much like the training side of my blog, I plan on discussing tips, tricks, resources and my experiences diving head first into Web Development. Wish me luck!

God, yet another Blog?

Yes, yes, it’s been done I know. How many food enthusiasts, triathletes, or  software engineer blogs are out there? Millions, but I don’t know how many combinations exist with all three topics. So why do it? For me of course! I plan on using this to keep track of cool recipes, interesting technologies, and most importantly knowledge I’ve gained in training for triathlons. I hope to include cool workouts, share experience about training with a full course load, tips for recovering, and anything else a new, but dedicated triathlete might find useful.