One thing that has helped me is journaling about the problem. I don't maintain a separate journal a al "Dear diary, today I got stumped... "
Instead I jot down my struggles along with the normal notes that I maintain about the projects I'm working on. This has often lead me to realize incorrect assumptions or discover avenues that were previously missing. It also makes my notes more interesting to read.
What really works for me is priming my brain with a problem (doing the research leg-work if you will), then taking a break. Going for a walk or some other menial task seems to work best.
During this time the problem will fester in my mind, ideas will shape up and fall down. It's important not to get distracted, this is an active-ish process.
When you come back home (very important is to at least the room where you started) you can begin putting words down on paper, or graphs or whatever, with a possible solution. This makes your thoughts really crystalize.
After that you really do have to try it out in practice. A lot of things that look great in your mind don't end up working as well within the constraints of The Real World.
Relevant:
"Putting creativity on a pedestal can also be an excuse for laziness. There is a lot of cultural belief that creativity comes from inspiration, and can't be rushed. Not true. Inspiration is just your subconscious putting things together, and that can be made into an active process with a little introspection.
Focused, hard work is the real key to success. Keep your eyes on the goal, and just keep taking the next step towards completing it. If you aren't sure which way to do something, do it both ways and see which works better."
-John Carmack
If you're a mortal, don't put too much weight in the advice of the gods.
People at the top of their game are usually too prone to attribute their success to hard work. People at the bottom are too prone to write it off as a question of genius.
The reality is somewhere in between: Carmack wouldn't be Carmack without a lot of hard work, but most people could never be Carmack no matter how hard they work.
He talks of letting your mind "rest against the problem" rather than battering at it. Crucial are time and space, and a little bit of time allocated for just letting your head clear out all the day-to-day crap that's in there.
There was an article on HN about genius and if it could be replicated and one point was to just look at a problem in a way you normally wouldn't. Well, that's kind of hard since I'm me and I look at things how I look at them. While I'm great at connecting unrelated ideas, it seems I'm not exactly great at problem-hacking. http://www.creativitypost.com/create/how_geniuses_think
While not exactly as concrete of a plan, there's the Anthony Robbins idea of living with expectancy (ie, expect the next page you turn, door you open, person you find yourself beside to potentially "open the flood gates"). http://i.imgur.com/VE1Mg.png I've been trying this the last few weeks and while it feels good to think about, I'm finding so far that it doesn't work (for me). Maybe I rationally don't expect 'expectancy' to work...
When it comes to group problem-solving, actually, this seems to be a benefit of using online collaboration instead of in-person. When people's mouths aren't running, there are less things proposed, and they are overall higher quality. (http://dl.acm.org/citation.cfm?id=159763)
Yeah, I can go along with this. A long time ago I discovered I can spend some time looking at a problem then just drop it and answers or avenues of inquiry to try suddenly pop up at the weirdest of times over the next few days. I can't imagine it works for everyone but "sleeping on it" works like gangbusters for me.
"Sleeping on it" also works for me as well. Often I'll think of a super difficult problem right before going to bed intentionally so I can get my subconscious working on it through the night. It works wonders. The next morning I usually have some insight... maybe not the whole solution but progress.
One more significant point is that you need to really cram all the requirements into your brain. And I mean all of them.
And secondly you'll only get a first order solution via this method. It certainly won't be elegant. If it ps an software engineering problem there will still be a lot of engineering left.
Rich talks about how thinking about something transfers the idea from your conscious mind to your unconscious mind (your big brain) where the real horsepower is. In another talk he mentions that he has been able to spend a year doing this three times in his life: one was for Clojure, one was Datomic, and one other yet-to-be-named project.
Transferring the idea" from your conscious mind to your unconscious mind"... that's exactly what I'm talking about. I've got to watch Rich Hickey's talk. Thanks for the link!
>This past week I was faced with a difficult problem. A really difficult problem... I kept going and in another hour or two, I was getting epiphany. I had the solution to my really difficult problem.
I am happy that he solved his problem, but was it really that difficult if it could be solved in just a few hours? How long would it take him to solve the most difficult problem in his domain?
In my experience it might take as much as a week even to identify the relative difficulty level of hard problems, which tend to be refined, restated in a simplified context, expanded out to boundary cases etc. To me it seems like "just looking at it" is the very first step in a longer process.
OP here. Well, it just worked out that I found my "solution" to my problem that afternoon. But I was thinking/researching the problem for many weeks at different levels. And I was finally coming to a deadline.
I agree though, it takes a long time sometimes to solve really difficult problems.
OK, I tried to suppress it but I'm letting it out.
I got tired of all the articles on doing something in a specific way. It's far too personal and everyone of us has the mental capacity to overcome serious problems with a solution of his own. A solution which, IMHO, can't be documented since it's just how our (individual) brain works and solves real problems.
The "Do Something in this way and you should benefit" is far too overrated, since it's one person's perspective and in no way should be taken too seriously, other than a short "advice" or a story to read before going to bed.
Therefore, I see no real value in posts like this, other than the editor's need to "let it out".
1. Brains are similar. How similar? We don't know until we tell someone else about what our own brain is doing, and find out that their brain works the same way. Some things might only work for one person. Some might work for 20% of people. Some might work for 99% of people. Nothing works for everyone, of course. Are you imagining that, instead of articles like this, we only see articles about things that work for everyone? That would be an empty set.
2. Anecdotes are meant to be used as grist for Bayesian updating. What I personally gleaned from this article was a slightly heightened confidence that your brain works at a certain speed whether you are pushing it to work or not, or worrying over its success or not, or whatever else--so it's better to just allow your brain to work, than put yourself in emotional turmoil over its need to find the solution. The technique itself (though see #1) isn't the important part of the article; rather, it's the knowledge found through the use of the technique--even though it was only implicitly stated!--that made the article valuable. The technique was only relevant in the way that a description of the experimental process and apparatus is relevant to a scientific paper: for rigor and replication of the original discovery.
I don't understand this response. Just because any individual such post may not work for you, if you read 100 such posts, you might come across a technique that works spectacularly for you. Much of life is about trial and error to discover what one is good at and how one is good at it. Hearing about what others have discovered in this regard cannot hurt our individual searches.
I guess that's the same reason solutions pop into your head after a day or so without even thinking about the problem. Your unconscious keeps working on it. I've read somewhere that this part of the brain is much faster than the part that actually does analytical thinking, since it deals with primitive functions and signal processing.
Yeah, your unconcious brain is performing FFTs like mad on your visual and auditory inputs with out effort but try to do an FFT with the thinking part of your brain and you're hooped with out extensive training in maths, and an uncanny ability to compute.
I think this hack is supported by a recent study (http://www.nature.com/news/why-great-ideas-come-when-you-are...) that working on a problem, then doing an unrelated activity that allows your mind to 'wander' can have measurable improvements to creativity when you return to the original problem. The author of this article seems to have stumbled upon some variation of this technique but I suspect that he could get better results if he applied the strategy more systematically.
Ahh..the good old "Let us step back and stare at the problem" trick that my learning theory professor used. Not sure what is particularly profound about it, external observation of it always (to me at least) made it sound somewhat mystical. I feel like the "hack here" is to have so much intuition for the problem that you defocus slightly and approach the problem using your "peripheral vision" (Using terminology from Paul Zeitz's excellent book on problem solving :).
Maybe it's obvious. But for me it was helpful to consolidate my notes and then to lower my expectations of "actively" finding a solution. But rather letting my brain stare at my notes for a long time as a means to more passively find a solution. The hack gives me permission to relax and let my brain or subconscious do its work.
I don't know... I spend significant amount of time with paper[1] when tackling difficult problems and I haven't seen similar behaviour in coworkers in any programming job I was at, so I sometimes keep wondering if it isn't my mind that's broken - everyone else seems to do fine just sitting at their computer and coding all day, while I have to do regular breaks for paper and/or whiteboard.
[1] - for the last few days I've been trying edw519's style of work with printing out listings of source code and going somewhere away from computer to plan next day's work. So far it's working quite well.
No, what the article seems to be talking about is almost exactly the opposite of what you are saying. Yes, getting organized and focusing on a problem is a good start but the article is talking about a way to get creative ideas when facing hard problems. Specifically, the idea is not to focus on a problem directly. It is sort of a lateral thinking strategy where instead of attacking the problem by banging down the front door, you instead try to find a side entrance. Last week, I was working on a very tough puzzle and trying to attack it directly with just pure concentration wasn't working. I just kept getting into a loop and couldn't make any further progress. The solution didn't come until the next day after I had stopped thinking about the problem and involved a very different strategy. Another thing I've noticed is that solutions that come this way are almost always more elegant than solutions that come from raw conscious effort, even when both methods solve the problem.
I agree that the practical steps the article proposes aren't mind-blowing.
On the other hand, if you look at the title of the post and the way the issue is framed:
>In fact, it was painfully difficult. And I was struck with a flood of emotions: fear, anxiety, despair, etc.
I just wanted to sleep and not deal with it.
I think it's pretty clear that what you're "hacking" isn't the task, its the anxiety. Sure, there are people out there who don't get overwhelmed by the challenges that they face.
But if you've ever spent time tutoring, you know that it is remarkable how often your charge has not presented his or herself with the facts of the matter at hand and reviewed what they already know. If you're in a frame of mind where you are procrastinating out of anxiety, telling yourself "just look at it" can lower the bar and make it possible to move forward.
I'm very surprised there are no responses to this article suggesting looking for relevant publications in the literature.
Often, difficult problems are not unique, and it can be very helpful to take a look at what has been published to see if anyone has already devised an elegant solution for the class of problems you are currently trying to solve.
Never had a problem letting my brain come up with solutions of its own. The problem lies in that my brain often comes up with solutions that are overly complex, which take a lot of work. The headache comes from trying to distill those raw ideas down into something more manageable.
[+] [-] raheemm|13 years ago|reply
Instead I jot down my struggles along with the normal notes that I maintain about the projects I'm working on. This has often lead me to realize incorrect assumptions or discover avenues that were previously missing. It also makes my notes more interesting to read.
[+] [-] dave1619|13 years ago|reply
[+] [-] Swizec|13 years ago|reply
During this time the problem will fester in my mind, ideas will shape up and fall down. It's important not to get distracted, this is an active-ish process.
When you come back home (very important is to at least the room where you started) you can begin putting words down on paper, or graphs or whatever, with a possible solution. This makes your thoughts really crystalize.
After that you really do have to try it out in practice. A lot of things that look great in your mind don't end up working as well within the constraints of The Real World.
[+] [-] Heinleinian|13 years ago|reply
Focused, hard work is the real key to success. Keep your eyes on the goal, and just keep taking the next step towards completing it. If you aren't sure which way to do something, do it both ways and see which works better." -John Carmack
[+] [-] bermanoid|13 years ago|reply
People at the top of their game are usually too prone to attribute their success to hard work. People at the bottom are too prone to write it off as a question of genius.
The reality is somewhere in between: Carmack wouldn't be Carmack without a lot of hard work, but most people could never be Carmack no matter how hard they work.
[+] [-] soup10|13 years ago|reply
[+] [-] zwischenzug|13 years ago|reply
http://www.youtube.com/watch?v=VShmtsLhkQg
He talks of letting your mind "rest against the problem" rather than battering at it. Crucial are time and space, and a little bit of time allocated for just letting your head clear out all the day-to-day crap that's in there.
[+] [-] personlurking|13 years ago|reply
While not exactly as concrete of a plan, there's the Anthony Robbins idea of living with expectancy (ie, expect the next page you turn, door you open, person you find yourself beside to potentially "open the flood gates"). http://i.imgur.com/VE1Mg.png I've been trying this the last few weeks and while it feels good to think about, I'm finding so far that it doesn't work (for me). Maybe I rationally don't expect 'expectancy' to work...
[+] [-] JoshTriplett|13 years ago|reply
[+] [-] gracenotes|13 years ago|reply
[+] [-] unknown|13 years ago|reply
[deleted]
[+] [-] tom_b|13 years ago|reply
I feel guilty about it - it's almost like being a monkey at the keyboard. "Will this push give me a pellet?"
But occasionally I do get the damn pellet . . .
[+] [-] petercooper|13 years ago|reply
[+] [-] dave1619|13 years ago|reply
[+] [-] keeptrying|13 years ago|reply
And secondly you'll only get a first order solution via this method. It certainly won't be elegant. If it ps an software engineering problem there will still be a lot of engineering left.
[+] [-] espeed|13 years ago|reply
Rich talks about how thinking about something transfers the idea from your conscious mind to your unconscious mind (your big brain) where the real horsepower is. In another talk he mentions that he has been able to spend a year doing this three times in his life: one was for Clojure, one was Datomic, and one other yet-to-be-named project.
John Cleese also talks about this in his lecture on creativity (http://www.youtube.com/watch?v=VShmtsLhkQg).
[+] [-] dave1619|13 years ago|reply
[+] [-] gajomi|13 years ago|reply
>This past week I was faced with a difficult problem. A really difficult problem... I kept going and in another hour or two, I was getting epiphany. I had the solution to my really difficult problem.
I am happy that he solved his problem, but was it really that difficult if it could be solved in just a few hours? How long would it take him to solve the most difficult problem in his domain?
In my experience it might take as much as a week even to identify the relative difficulty level of hard problems, which tend to be refined, restated in a simplified context, expanded out to boundary cases etc. To me it seems like "just looking at it" is the very first step in a longer process.
[+] [-] unknown|13 years ago|reply
[deleted]
[+] [-] dave1619|13 years ago|reply
I agree though, it takes a long time sometimes to solve really difficult problems.
[+] [-] sheldor|13 years ago|reply
I got tired of all the articles on doing something in a specific way. It's far too personal and everyone of us has the mental capacity to overcome serious problems with a solution of his own. A solution which, IMHO, can't be documented since it's just how our (individual) brain works and solves real problems.
The "Do Something in this way and you should benefit" is far too overrated, since it's one person's perspective and in no way should be taken too seriously, other than a short "advice" or a story to read before going to bed.
Therefore, I see no real value in posts like this, other than the editor's need to "let it out".
[+] [-] derefr|13 years ago|reply
2. Anecdotes are meant to be used as grist for Bayesian updating. What I personally gleaned from this article was a slightly heightened confidence that your brain works at a certain speed whether you are pushing it to work or not, or worrying over its success or not, or whatever else--so it's better to just allow your brain to work, than put yourself in emotional turmoil over its need to find the solution. The technique itself (though see #1) isn't the important part of the article; rather, it's the knowledge found through the use of the technique--even though it was only implicitly stated!--that made the article valuable. The technique was only relevant in the way that a description of the experimental process and apparatus is relevant to a scientific paper: for rigor and replication of the original discovery.
[+] [-] nessus42|13 years ago|reply
[+] [-] psykotic|13 years ago|reply
[+] [-] dave1619|13 years ago|reply
[+] [-] hndl|13 years ago|reply
[+] [-] alexmic|13 years ago|reply
[+] [-] fleitz|13 years ago|reply
[+] [-] hexagonc|13 years ago|reply
[+] [-] eshvk|13 years ago|reply
[+] [-] keeptrying|13 years ago|reply
[+] [-] dave1619|13 years ago|reply
[+] [-] TeMPOraL|13 years ago|reply
[1] - for the last few days I've been trying edw519's style of work with printing out listings of source code and going somewhere away from computer to plan next day's work. So far it's working quite well.
[+] [-] linker3000|13 years ago|reply
This is a 'hack'? Looks more like common sense and avoiding procrastination to me.
I know this sounds snarky, but attributing the word 'hack' to this is just sexing up something people have been doing since forever.
Edit: http://en.wikipedia.org/wiki/Heuristic
[+] [-] hexagonc|13 years ago|reply
[+] [-] bgilroy26|13 years ago|reply
On the other hand, if you look at the title of the post and the way the issue is framed:
>In fact, it was painfully difficult. And I was struck with a flood of emotions: fear, anxiety, despair, etc.
I just wanted to sleep and not deal with it.
I think it's pretty clear that what you're "hacking" isn't the task, its the anxiety. Sure, there are people out there who don't get overwhelmed by the challenges that they face.
But if you've ever spent time tutoring, you know that it is remarkable how often your charge has not presented his or herself with the facts of the matter at hand and reviewed what they already know. If you're in a frame of mind where you are procrastinating out of anxiety, telling yourself "just look at it" can lower the bar and make it possible to move forward.
[+] [-] natekupp|13 years ago|reply
Often, difficult problems are not unique, and it can be very helpful to take a look at what has been published to see if anyone has already devised an elegant solution for the class of problems you are currently trying to solve.
[+] [-] demione|13 years ago|reply