Tuesday, October 25, 2016

Upgrading a Netgear Orbi - Solution to "This firmware file is incorrect!"

I just got a brand new Netgear Orbi router with extender (called a satellite) this weekend.  When navigating to the firmware upgrade page, the UI indicated that I had an upgrade available for the base station, but the satellite unit was stuck in "please wait" mode.  So, I decided to try upgrading the firmware manually.  I read online that you should always upgrade the satellite first, so I downloaded the firmware for both, logged into my router, and attempted to upgrade.  I was then presented with the following error message when attempting to load the firmware for the RBS50 (the satellite):

The firmware file is incorrect!  Please get the firmware file again and make sure it is correct firmware for this product.
After a little poking around, I realized that my satellite appears in the list of "attached devices" and when I connect directly to the satellite, there is a different interface for upgrading the firmware of that device.  So, I used that interface to upgrade just the satellite (the RBS50).  Once finished, I could upgrade the firmware of the Orbi router using the automatic upgrade feature.

Thursday, June 30, 2016

Why I'm Sticking With Evernote

I have been a freeloader with Evernote for since 2012 when Google Notebook shut down until this year.  A few months ago, Evernote forced me to upgrade to a $25/year plus account to re-enable the ability to send notes to myself via email.  I paid up for a single year and deferred really thinking about switching to something cheaper.  It turns out that I don't use the email-to-self-feature much so shortly after paying, I decided to go back to freeloading once the year was up.  Then, the news hit that Evernote was going to limit the free tier to two devices and raise the price from $25 to $35 for the lowest paid tier.  I have 6 devices (counting two Chromebooks that have the Evernote Android app installed).  My Evernote notebook is about 1/2 Gigabyte today and growing.

~/Library/Containers/com.evernote.Evernote$ du -cksh 469M . 469M total

$35/year seems like a lot for an app, but self-hosting only the storage would exceed that cost on any platform that supported running my own applications.  I wouldn't self-host a server as important as a repository of my personal knowledge.

What about Google Keep?
I was burned pretty hard by Google when Google killed Google Notebook and then launched Google Keep a few years later and no ability to transfer my data, so I want to keep my notes with a company that takes my long-term data seriously.  I still don't think it wise to store my notes with Google even for free.  Even though it does everything I want from a note-taking app.  Evernote even alludes to this fact in the marketing material around this change.

Evernote isn’t a vast corporation, and note-taking isn’t a sideline for us. It’s what we do, and we strive to do it better than anyone else. We hope you’ll continue to capture your thoughts and develop your ideas with us. [source]

Microsoft OneNote?
I've been burned so many times by Microsoft's embrace and extend, it's going to take a long time before I can be sure Microsoft would actually support all of the platforms I want to be able to access my data.  I'd rather trust Google.

SimpleNote, Todo.txt, Other?
I really like the ability to take a picture of my hand-written notes and then search for them.  I'm willing to pay for this feature.  I'm not aware of another cloud-based service with this feature other than these.

Sunday, August 2, 2015

Book Review: Inspired by Marty Cagan

This book is a collection of blog posts cobbled together into a book that has no real thesis and often contradicts itself.  This contradiction is dangerous because it emphasizes putting creative control of the product in the hands of those with titles containing "UX" and "Product" and devalues the creative and innovative capabilities of the rest of the orginization.  Due to this centralization of roles and internally inconsistent nature, it could be treated like a religious text where the high priests of Product can cherry-pick whatever ideas they want to build any product they imagine so long as they can get customer to say it makes them happy.

The advice in this book isn't bad, per se, it just isn't great.  The author, Marty Cagan, takes the waterfall model of product design and extends it into product, defining a dozen or so different specific job titles and outlining when those titles are necessary in what phases of a project.  Unfortunately, in a highly creative and collaborative environment, this advice could potentially slow down development of a product and stifle the creativity of people who can do more than exist in their pre-defined roles.  Worse, he doesn't appear to understand the creativity of engineering and what kind of innovation takes place in the technical side of the product development lifecycle.

Cagan provides some excellent advice.  Namely:
  • Product owner sees the product from start to finish and participates in all user testing.
  • The Engineering Lead is involved in the entire process to provide information about what is possible and how hard it is.
  • Focus groups are bad because customers don't know what they want.
  • Sometimes the customer asks for x and the best thing for the business is to give them y.
  • Don't charge customers who are in the beta.
  • Don't lead the witness.
  • Don't confuse customer requirements with product requirements.
I've personally seen the benefit of following these rules and have seen that NOT following these rules can lead to direct negative consequences.  

Let's take a look at the closest thing I could find to a short thesis in the way of a summary of how to better run a startup.  It's basically a copy/past of this blog post into the book.
Here’s how it typically works. Someone with an idea gets some seed funding, and the first thing he does is hire some engineers to start building something. The founder will have some definite ideas on what he wants, and he’ll typically act as product manager and often product designer, and the engineering team will then go from there. The company is typically operating in “stealth mode” so there’s little customer interaction. It takes much longer for the engineering team to build something that originally thought because the requirements and the design are being figured out on the fly.
After 6 months or so, the engineers have things in sort of an alpha or beta state, and that’s when they first show the product around. Things rarely go well in this first viewing, and the team starts scrambling. The run-rate is high because there’s now an engineering team building this thing as fast as they can, so the money is running out, and the product isn’t there. Maybe the company gets more funding and a chance to get the product right, but often they don’t. Many startups try to get more time by outsourcing the engineering to a low-cost off-shore firm, but it’s essentially the same process with the same problems.
Here’s a very different approach to new product creation, one that costs dramatically less and is much more likely to yield the results you want. The founder hires a product manager, a product designer, and a prototyper. Sometimes the designer can also serve as prototyper, and sometimes the founder can serve as the product manager, but one way or another, you have these three functions lined up - product management, product design, and prototyping – and the team starts a process of very rapid product design and iteration.
I describe this process in detail in “How To Write a Good PRD,” but there are two keys: 1) the idea is to create a high-fidelity prototype that mimics the eventual user experience – it’s just fine if the back-end processing and data is all fake; and 2) you need to validate this product design with real target users.
In this model, it is normal to create literally dozens of versions of the prototype – it will evolve daily, sometimes with minor refinements and sometimes with very significant changes. But the point is that with each iteration you are getting closer to identifying a winning product. This process typically takes between 3 weeks and 2 months, but at the end of the process, you have a) identified a product that you have validated with the target market; b) a very rich prototype that serves as a living spec for the engineering team to build from; and c) you now understand at a much greater degree what you’re getting into and what you’ll need to do to succeed.
Now when you bring on an engineering team, they’ll start off with a tremendous advantage – a clear understanding of the product they need to build and a stable spec – and you will find that the team can produce a quality implementation much faster than they would otherwise.
This model of prototype-based product experimentation is increasingly becoming the norm in the manufacturing world, but for some reason this hasn’t taken off in software. I think we’re such an engineering-driven culture that we just naturally start there. But any startup has to realize that everything starts with the right product – so the first order of business is to figure out what that is before burning through $500K or more in seed funding.
I believe this model applies beyond startups to much larger companies as well. The difference is that bigger companies are generally able to underwrite the several iterations it takes to get to a useful product, but startups often can’t. But there’s no reason for the inefficiencies that larger companies regularly endure.
Allow me to nitpick this idea.
  1. Stealth mode is bad.  Cagan and I agree that stealth mode deprives the team of feedback, but another solution to this problem is to just not do stealth mode.
  2. It takes longer to develop than originally thought. This is a widespread problem related to an engineering team's poor estimation skills.  Skills can be improved.  A typical Agile engineering team is bad at this for the first few iterations of its existence but can quickly refine those estimates as it retrospects on those estimates after each iteration.  My engineering team is typically off estimate by roughly 10% or less, and is quick to refine estimates as we discover new things.  Sometimes we discover an open source tool that lets us re-use rather than rebuild and the estimate drops, and sometimes we learn from our users that we are building the wrong thing and must adjust accordingly.  But, I argue that the total time to market is faster this way than prototyping without engineers first and then bringing in engineers - something I elaborate more on shortly.
  3. Six months to the first beta.  Cagan and I agree that spending six months without any user feedback is bad.  But, another solution is to just develop the most important thing first and start the user testing against the actual product rather than a prototype.  My engineering team typically has a beta of the core functionality of a new product ready for user testing after two to four weeks.  I'll get into why this is optimal shortly.
  4. High Run Rate.  I suspect that the run-rate wouldn't be that much higher to prototype using real engineers building a real product rather than paying for prototypers to build something that will be thrown out when the engineers show up and tell you the prototype needs to be re-designed because it didn't take into account technical costs.  In fact, I would bet that the total cost to get to market with Cagan's solution to this problem would be much higher than if a real engineer worked on the prototype because UX innovation would be taking place in parallel with engineering innovation rather than delaying the engineering innovation until after the UX design was considered "done" (something that's not totally possible until the engineering innovation has taken place anyway).
  5. High Fidelity Prototype.  In the Agile world it would not be easy to create a high-fidelity prototype without consulting with the engineers about what is technically possible for the first version.  At best, this prototype is a close approximation to the perfect world given unlimited engineering budget.  Remember, given a complete waterfall-style specification, you're going to spend far more than 80% of your engineering effort on far less than 20% of the product and without engineers present for the prototyping phase, small changes to that 20% could have a dramatic effect on that 80% of the budget.  I'm not arguing against the high-fidelity prototype.  I'm arguing that the engineers should be building that prototype rapidly using test-driven development, and with very short (sub-weekly) iterations and frequent (sub-weekly) releases.
  6. 3 weeks to two months.  In the solution I propose, after those 3-8 weeks, you not only have yourself a high-fidelity prototype, but you have yourself a few engineers who are intimately aware of the customer's needs, and a suite of tests that document the specification of the user-facing part of the application.  Swapping out the fake backend with something that contains security and persistence is simply the next phase of the already-in-progresss project.
  7. Clear understanding and a stable spec.  If this were a waterfall-style shop, Cagan makes a good point.  If you want the engineers to passively consume the specification and build a system that meets that specification, Cagan proposes a great solution to the outlined problem.  But, if this system were being built by true software craftsmen, then they would look at the specification and begin to suggest small changes that could drastically reduce the cost of the system.  Those proposals would need to be run through the same UX design process and suddenly that 3-8 weeks spent in pre-planning starts not to be so valuable.  What's missing is that the UX and Product team do not have a clear understanding of what makes sense to build technically until the engineers receive and respond to that specification.  And, when that starts to happen, that specification will no longer be stable.
  8. Prototype-based experimentation hasn't taken off in software.  The fact that the author has no idea why the idea hasn't taken off in software strikes directly at the heart of what is wrong with the thesis of this book.  The author has little to no understanding of the state of the art in software engineering processes.  Lean Startup, Scrum, Kanban, and the host of other Agile philosophies have their roots in the manufacturing world and describe themselves as ways to create small "experiments" or continuous evolution of a prototype to create software.  Software engineering is exactly the thing Cagan is proposing we do, but he's proposing we do it without the software engineers.  I suspect this has something to do with the companies he has worked for: HP, AOL, Netscape, eBay - companies that aren't known for their ability to rapidly innovate through software engineering.  Earlier in the book he advises not to make the actual product the prototype (copied from this blog post) because costs to build test and deploy are too high, but another solution to this problem is to crush those costs down so this is no longer a barrier - something that we have done fairly well at the company I am working for now.
I think #7 and #8 are the most important problems with Cagan's thesis.  First, it's unfounded and inaccurate, and second, it's a hypothesis that the author himself hasn't appeared to have tested (or at least he doesn't state he has any experience testing this theory).  Finally, it appears to contradict advice he gave earlier in the book and taken from a different blog post:
Maybe the biggest missed opportunity in [old-school product management], is the fact that engineering gets brought in way too late. We say if you’re just using your engineers to code, you’re only getting about half their value.  The little secret in product is that engineers are typically the best single source of innovation, yet they are not even invited to the party in this process.
Which I whole-hardheartedly agree with, speaking as a self-promoting engineer.

Cagan measures the success of a product based on customer happiness, but he never shows how to measure happiness.  In fact, he never shows how to measure anything.  The word data-driven doesn't appear in the book once.  Granted, a very important way to measure the success of a product is to simply ask customers if they are happy, but even Cagan conceeds that simply asking customers isn't the best way to get requirements from them.  Early on in Google's life, they tried to sell their search technology to Yahoo.  Yahoo didn't want to pay for it because search was considered a solved problem and users already loved Yahoo.  Google had measurably better technology, but no users loved Google, so Yahoo passed.  Upon launching, Google didn't simply ask users if Google Search made them happy.  It measured every time people navigated to the second page of search results and considered that click a "failure" of the search.  Google set a goal to reduce the average number of "next" clicks per search every year.  This allowed Google to innovate in a scientifically measurable way and stay ahead of the competition even though users already loved Yahoo.  True innovation isn't simply putting a fresh coat of paint on a mediocre product or giving the user a horse that's easier to ride.  True innovation happens when creative people solve an important problem using a solution everybody else thinks is impossible or dumb.  The best innovation, rarely developed through UX, is disruptive.  The first automobile was considered slow, difficult to operate, and expensive to maintain compared to the horse.

Cagan's approach is a great way to take a waterfall organization building mediocre products to another level.  But, he isolates the creative process to the few people with the right job title and gives them the tools of subjective psychology-driven decision-making.  The customer may end up loving your product, but it won't necessarily be the best product you could have built.

Wednesday, February 11, 2015

Functional Programming and Domain Driven Design

I was in a meeting full of senior engineers a few months ago, and we were discussing the state of our Javascript which lead to the topic of Functional Programming.  I mentioned that I thought we were spending a lot of time writing packages of functions that were fairly composable, but we were having trouble reducing duplication because our code was organized by project, specific to the problem being solved at any given time, with no concept of problem domains.  I suggested we apply the principals of Domain-Driven Design to raise the level of abstraction and reduce duplication.

Unanimous laughter followed.  Buried inside the laughter was the comment, "DDD?  What is this, 2008?"

I assumed that I was missing something and that the concepts of DDD were incompatible with FP because all of the core "things" described in DDD have the word "object" in their name.  But, while DDD as described by Eric Evans is implemented in an Object Oriented language in his book, it isn't implied by the fundamental approach.

Then, I stumbled on this video which hypothesizes that DDD is even better with FP.  The speaker asserts that there isn't any reason Functional Programs can't also contain a "ubiquitous language", "bounded contexts", etc.

He doesn't exactly prove the hypothesis.  But I assert that DDD, by my definition, is compatible with FP.  So, below is my attempt to map each DDD buzzword to something that is easy to implement in a Functional program.

Value Objects - Immutable Values are all Value Object in FP.
Entities - Giving a Value Object a specific ID makes it an Entity.  Easy in FP.
Aggregate - A collection.  Functional program is full of these.
Aggregate Root - A function that maps over an Aggregate.  I suppose we can use types to protect our Aggregates from external mutation, but this wouldn't be a necessary if we kept all of our data immutable.
Domain Event - Fans of Big Data will likely find the concept of mapping and reducing over raw data, a very Functional idiom.
Service - Functions that operate on data could be considered services.
Repository - In the process of isolating non-pure database access from pure functional code, programmers often use some sort of repository pattern to abstract the database access behind a composable API.

What does this mean? It means that we OO expats, shouldn't throw out everything we were doing to organize our domain.  We should be taking the most important and useful OO tools with us as we assimilate into our new FP world and only deviate once we have proven a superior idiom is available to solve our problems.

Sunday, November 3, 2013

Amazon Kindle: Resetting "Last Page Read"

Somehow, a book that I had just started reading on my kindle jumped to the end.  From that time, all of my Kindle devices showed that the "farthest point read" was that point, and whispersync was effectively broken for that book.  Here's how I was able to reset the "last page read" for that book:
  1. Go to the device where you have the book at the location where you want it to be set (either at the beginning of the book or at a given page) 
  2. On http://www.Amazon.com click My Account > Digital Content > Manage Your Kindle. 
  3. Click "Whispersync Device Synchronization". 
  4. Turn Synchronization Off
  5. Go back to your Kindle device and "Sync to furthest page read".
  6. Remove the book from that device.
  7. Re-download the book from you "cloud".  On the kindle, go home and tap "cloud" to find the book.
  8. Open up the book.
  9. Go back to "Whispersync Device Synchronization"  and turn it back on


Tuesday, October 29, 2013

Google Is Slowly Closing Android

Ars Technica recently posted an article that outlines various ways Google is closing. I think the majority of the article is spot-on, and for the first time, I am seriously disappointed in the Android Open Source Project leadership for allowing this to happen.

If this policy of leaving the OSS project to stagnate continues, I will likely investigate an alternative to Android as my primary operating system.  My hope is that projects like Cyanogenmod can take up the leadership role, or at least threaten Google, such that Android doesn't become completely closed source.

Friday, October 11, 2013

Fixing Lag On The Barnes And Noble Nook HD+ Running Cyanogenmod 10.2

UPDATE: CyanogenMod 10.2.1 has trim enabled. Don't follow these directions. Just install 10.2.1.

I have a Nook HD+, and the only way to love this tablet is to put CyanogenMod on it, because that turns it into a very-near stock android tablet - no special Barnes & Noble stuff, and it gets really really fast.

This tablet has one huge flaw.  The memory controller has a tendency to fail when the system runs a trim operation, something that all modern Android devices have that helps keep them from slowly starting to lag more and more over time.  Here is a decent description of what trim is and why you want to have it on this device. The solution until now, even on the stock HD+ from what I can tell, is to just disable trim completely.  So, what happens is that, regardless of if you are running the stock firmware from Barnes & Noble, or if you have upgraded this tablet to CyanogenMod 10.2, this tablet will progressively get slower and slower until it is nearly unusable.

But, there is a solution, if you are willing to to take a risk.  Apparently, the Nexus 7 has the same memory controller as the Nook HD+, and Google has patched Android to fix the bug.  There hasn't yet been enough confirmation that this is fixed, so it isn't in any mainstream kernels that I can find.  But, if you are running CyanogenMod 10.2, you can flash this kernel right over your CyanogenMod 10.2 nightly and trim will be enabled.  Once it's enabled, you can run this app to fix the lag.  After running the app once, it'll probably be a while before it needs to be run again, so in case there IS a bug, just re-flash CyanogenMod Nightly to put the original kernel back.

So, to recap the steps to make the Nook HD+ not lag when running CyanogenMod 10.2:
  1. Download the patched kernel zip.
  2. Install the lagfix app.
  3. Use CyanogenMod updater to get the latest nightly ready to go.
  4. Reboot into ClockworkMod.
  5. Flash the new kernel and reboot.
  6. Run the lagfix app for all partitions.
  7. Reboot into ClockworkMod again.
  8. Flash the latest nightly zip (which will swap in a kernel without the fix for protection).
  9. Enjoy Fast Tablet.


Bookmark and Share