My earlier comment might seem off-putting, so here is the text of the article linked in it, to provide a context for the discussion.
Robb Seaton has written an excellent, must read article on the “The Science of Problem Solving.”
In the quest to optimize the process wherein the solver is happier most of the time and is able to do it more quickly, I observed that the approach could be further refined. We can and we should develop a better strategy.
So, here we go.
Key challenge
-------------
The article proposes two ideas:
Leverage the sleep time for the incubation, and
Think about the problem in different moods
These ideas are plausible. However, they suffer from a steep cost in terms of the time invested for reaping the “aha!” moment.
To discuss the reason and a possible solution, let us introduce a variable called “protag,” short for the protagonist, so that we say that the “Protag wants to solve the problem,” rather than, “We want to solve the problem of improving the problem solving.”
Let’s begin analyzing the terrain. What are the minimum, always available tools for the protag? The triplet: the body, the mind and the intellect.
A few definitions
-----------------
What is sleep? A tight binding of the body and the mind.
What is a mood (happy or unhappy etc.)? A tight binding of the mind and the intellect.
We can get rid of a minor lemma: the necessity of the sleep. We agree it to be a necessity for obvious reasons. We acknowledge it and leave it because we want to explore another interesting dimension of this “problem solving” hypergraph.
Formulating the conjecture
--------------------------
Could there be a more effective approach, in which, lesser cost in terms of time and physical energy is spent, to improve the problem solving ability?
Search for an answer
--------------------
Let’s model the problem in terms of protag’s mind’s quest to walk out of the maze. The darker the maze, the scarier the problem.
For the mind trying to navigate through the dark maze, the intellect is the torch that illuminates. The brighter the torch, the easier to avoid obstacles and hence, the faster the solution. The candela of this torch depends on the mind’s ability to recall a specific pattern.
Since the mind can travel faster than light, the protag can optimize the effort required to boost the candela simply by a continual build-up of the pattern library. In other words, the protag need not worry about the mind facing the heavy burden of carrying the library in one hand and the torch in the other.
Why do the bindings matter?
---------------------------
During the navigation, the protag’s mind has reached a door it wants to unlock. It has summoned every single pattern to be applied for unlocking it. In the process, the mind (as algorithm) has exhausted the problem space. How does it react?
It does not know how to react!
In practice, the mind slips into the library of dark patterns and picks up the thing that is on the top of the stack.
It can be a feeling of fatigue, or it can be a tantrum.
The mind yells and throws an egg into an electric fan. Meanwhile, the poor protag, suffering from a tight coupling of the mind and the intellect as well as that of the body and the mind, might actually:
throw an egg into an electric fan,
take a drink,
sleep,
take a shower, or
take a stroll in the park — we all get the idea.
Clearly, to bring in a qualitative improvement, the protag ought to work on actively decoupling the body, mind and the intellect, before and during problem solving.
It can start with a realization, that “I am not my mind!” just as “I am not my pen!” Once this realization is internalized, the protag can liberate negative feelings (like “I am exhausted,” “I am angry”) from the stable of dark patterns; treating them more as blinking LEDs indicating exhaustion of the pattern library, rather than “dark walls.”
The protag can then say, “Oh, this LED says that I need to add more patterns in the library, so how about applying the intellect for synthesis of existing patterns. But, exactly how am I supposed to compose them?”
Good question.
By now, must have noticed that, instead of the mind leading the protag to a solution, what is happening is quite the other way? On a happiness scale, this approach looks better. May be, even wiser.
Enriching the library of patterns
We might say that the process for understanding is akin to parsing a new program. During parsing, the type information is generated and most of it is strongly typed as in Haskell. Now that the pattern is parsed, and does not exist in the pattern library, how to store in the library in terms of existing patterns in the library. Summon the Common Lisp and defmacro to the rescue. But, this defmacro must obey one rule:
Never violate type safety, i.e. 2 + “Hello I am an A” is a a big no-no.
Consequently, it should be no surprise that a well-chosen set of primitives, together with a good set of macros is especially valuable. In that sense, we can say that the acquiring knowledge is constant re-factoring of this “language of thoughts.”
Let P be the no. of primitive types and M be the no. of macros.
Then, the size of the library, denoted by L is:
L = P + O(M), in big-O notation.
Intuitively, P < M. But, M “grows”, for each defmacro.
To shrink “M,” the protag must re-factor the existing macros in the library by leveraging macro writing macros.
Essentially, this illustrates the difference between the smart and the smarter,
M(smart) > M(smarter) in numbers, but,
M(smart) < M(smarter) in enrichment due to re-factoring.
As L grows, P should grow O(N), while the growth of M must be tamed from O(N^x) to O(N^y.logN) for some N, x and y, such that x > y.
Building this all important library is more akin to growing a plant from the seeds. How does the protag water this plant?
In other words, how does the protag know that a new macro needs to be introduced?
Thinking about thinking will surely help, but only so much.
The protag can know that for a new macro must be introduced, only when its mind is faced with an unlocked door. Hence, building the library is a constant struggle: the protag must improve the candella by attempting radically new problems. It is going to take time, but lesser, compared to depending on the bolt from the blue.
However, as every seasoned Lisper knows: growing a library of macros, where many of them are macro writing macros, is hard. So, the protag must wander in different disciplines for inspiration: like Physics, Cognitive Psychology, Literature, Music, Maths, Poetry, Sculpture, Sports, and so many different pursuits.
Inspiration
-----------
Let’s simplify the definition of an inspiration. An inspiration is an external force that upsets the equilibrium in the protag’s universe, mandating a re-assessment of the library and the eventual struggle to re-factor the macros and in the rare case, the addition of a new primitive.
Wrapping up, let’s look at the pros and cons of this approach.
The pros
Greater accuracy a.k.a. qualitative improvement
Faster
As a nice side effect, finer control over emotions i.e. to be phenomenally happy even during the oddest of the odds
The cons
Conscious and systematic effort is required to develop the skill.
Where is the roadmap?
How does one assess the progress?
Where does one even begin?
Closing remarks
---------------
This merits further thought. But, since we now understand the problem better, we can work towards a better solution.
Thanks
------
I wish to convey a sense of deep gratitude towards the author of the article. The article left with interesting questions and the pursuit of the answers was both, a profitable and a pleasant outcome.
Re: sleep and moods, though a plausible idea, suffers from a tight binding problem. This is very costly in terms of time invested for reaping the "aha!" moment.
I sat down to elaborate the reason and a possible solution, but the word count went over 1100. So, here it is in all its glory:
If you meant database engine, then you should contribute to open source database projects like PostgreSQL or MariaDB. This assumes that you're comfortable with populating and maintaining databases.
Start with downloading the source code for say MariaDB, reading internals documentation on project Wiki, compiling it on your machine.
This is just amazing! It feels like suddenly creating a new market. It has the potential to change the way customer service is addressed in the field: for washing machines, data loggers and may be hundreds of devices.
Happy to learn that the post resonated with you. I appreciate it.
Once a VP at a customer told me that a diamond is just a piece of coal that shines under pressure.
The quest to become a jewel in the market can be a powerful initial thrust to work as a consultant. Combined with the opportunity to work in a diverse set of technologies and business domains turns to be a pretty compelling force.
Re: compensation, tptacek and patio11 have shared some great thoughts in the past.
Kind of agree with you and OP, so let me add to the discussion with the caveat that YMMV etc. Some of these points may overlap with comments by fellow HNers in these threads.
In any organization as a prospective customer, there are three forces at work that matter to a consultant: users, decision makers and finance folks. Ideally, the decision makers appoint a dedicated person (or team) who mediates users and finance guys to create a coherent road map for the system to be built. What tends to happen in practice is a combination of chance, human errors and conflicts in priorities.
To survive, we must understand ourselves as consultants, the terrain and the opposing forces (anything that prevents money getting deposited in the bank).
Consulting is a business => the individual consultant must shoulder quite a lot on himself:
* Advertising himself
* Finding work
* Meeting prospective customers
* Writing proposals
* Closing the deal
* Writing a contract
* Defining the scope of work
* Nailing what is to be done and what is to be avoided
* Constructing components of software
* (In worst cases) baby-sitting customer's devs / ops
* Testing his modules
* Getting money in his account
So, when we talk with customers, there is inherent pressure in closing the deal versus precisely defining everything. Icing on the cake: back of the mind, you might worry about milkman's bills and the birthday gifts to be brought for your kid's friends. Always have a good runway: six months seems optimum. Technically, even three months should suffice, but it is insufficient for the edge case when these three things hold true simultaneously:
* You've to walk away from a customer in the middle of a project, and
* Dry sales pipeline, and
* Unexpected crisis at the personal front: very close friend in need of money for surgery etc.
God be with you.
Now, back to the main track. The key is to first understand the priorities of stakeholders. The contradictions are easier to grasp once we put ourselves in the individual's shoes:
* [Convenience] The users want everything including the kitchen sink,
* [Accountability for the risk] They want it yesterday but won't commit to it on paper since you're in early sales stage,
* [Cost] The finance folks want stability and the most economical price,
* [Availability] The decision maker prefers to spend the least of her time to be spent on this activity, except when it's her pet project.
These roles may be played by different people or the same person.
Again, we don't have to agree, but acknowledge that things are meant to be imperfect. Most people are actually decent and rational. Now, somehow, the consultant must figure out a good path. To begin, he should have a good handle on:
* The industry in which the customer operates, e.g. BFSI, Energy, online marketplaces, food, fashion etc.
* The nature of the market: oligopoly, crowded, emerging et al.
* The level of Govt. regulation: HIPAA? PCI?
* The customer's position: among top tier, mid-size, small firms.
* Customer's customers: helps to track the wind.
* Standard pain areas in the market.
* How customer's competition and partners survive the pain, specifically the differences among the customer's approach and the competitor's ones.
* The financial health of the customer: if possible, try to talk with their existing suppliers in other arenas (e.g. the guy who supplies printer cartridges, the cleaning lady etc.).
Based on that, assuming we've landed a hot lead, say that we want to convert it to money. There will be a lot of conversations over the phone, e-mail, chats, face to face; what is to be left out of those conversations is really context sensitive. The important parts are:
* What problem we're going to solve for all stakeholders?
* How we're going to solve it?
* Communicating our approach and progress on a timely basis.
* Improving Vulcan skills: like money, expression of anger is a great servant but a terrible master.
I leave out the details on the development process, since it depends on the individual.
Here are a few things which worked for me marvellously in certain contexts and doomed in others:
* Setting expectations by explaining the cone of uncertainty in the estimation process. Always be prepared for some customers, who deliberately mislead you that they don't understand the mumbo-jumbo, projecting ignorance as a shield so that they can ruin you on payments. Mostly you should refuse such customers, unless you're in an extreme crisis on multiple fronts.
* Writing use cases for the modules / systems I was supposed to work. Most customers say that they don't want any time spent on "documentation," to which I reply that writing a document is not "typing" but "nailing the thoughts to deliver higher quality work."
* Creating mock ups using sketching tools (even Excel/LibreOffice can be good enough, if one is pressed too much for time) in the absence of UX designers on the team.
* Writing test cases before writing my code. Helps to ease the integration.
* Optimizing lazily: progressively refining the artefact (document / mock up / test / code).
* Knowing the expected value of each task:
(amount of money * satisfaction * improvement in reputation) / (e-mail tennis * stakeholder's understanding of requirements * payment latency * time spent)
Finally, reducing it to just one sentence: be nice, believe in yourself and humanity, be multi-dimensional and please heed Guy Kawasaki's advice about reading voraciously and omnivorously.
Since you're pretty much at design stage, two ideas are worth exploring:
1. Uniqueness typing (re: linear typing for mutable data) is a neat addition. The paper by John van Groningen et al seems the most appropriate: "Exchanging Sources Between Clean and Haskell: A Double-Edged Front End for the Clean Compiler."
With fish 2.0, the shell stores a list of "cd" commands issued from a specific directory, so mostly just typing "cd" yields an auto-completion hint out of the box.
Assuming that you've found an interesting business problem to solve:
1. Write a list of desired features. Next, prioritize those features.
2. Write brief functional specifications for features of highest priority.
3. Decide on how you want to use the software stacks for different tiers, e.g.
Browser - Bootstrap or Zurb, AngularJS
Server stack - node.js, Ruby, Python
Database - MySQL, Postgres
These examples are sketchy, just to give a feel of things.
4. Research any libraries that automatically handle common functional units, such as authentication and authorization, analytics et al.
5. Some functional units might be implemented more easily by gluing together services. E.g. using New Relic for deep application monitoring, Twilio for cloud communications.
6. Design and implement all functional units.
7. Profit!
8. Write a Thank you HN note, :)
One should do this exercise for a "toy" application like "To Do List" first. It helps remarkably when we go about building serious stuff.
Books:
1. Code Complete by Steve McConnell - worth at least skimming it.
It may look like a huge undertaking, but it really isn't. A quote from Michael Abrash seems appropriate here, "Winning a hard race prepares us for the next harder one."
https://github.com/jberkel/android-app.g8