Financial advising deals with the future, which "always in motion is", according to Yoda. (If you've got four minutes, follow that link. You're welcome.) We can maximize our chances of a good outcome, but the fact is that wise decisions and good outcomes don't always go hand-in-hand. Even the casino doesn't win every day.
This leads us to an interesting puzzle: how do we judge a financial decision apart from its outcome? If I invested all my money in my company's stock, and it quintupled over the course of a year, was that a wise decision? If I faithfully took out disability, homeowner's, auto, life, and disability policies, and never needed any of them, was that foolish? Most of us would intuitively say "no" to both of these, but if outcomes aren't necessarily a result of the quality of a decision, how are we to learn from experience?
How are we to make good decisions?
"Past performance does not necessarily predict future results"
The first step is simply to keep in mind the truth that wise decisions do not, in fact, guarantee good outcomes. (Similarly, foolish decisions do not guarantee bad ones.) Just because a decision led to a good outcome doesn't mean that it was the "right" one, nor vice versa. Our mammal brains have a hard time with this, because we're wired to continue to do the things that have gotten us good results in the past. That's why it's hard to sell a stock that's been going up recently, and why good financial advisers dread the words, "my first options trade made me so much money!"
So step one is to repeat the mantra, "wise decisions do not guarantee good outcomes." When you've successfully separated decisions and outcomes, you're much less likely to make (or continue!) poor decisions that just happened to have good outcomes in the past.
"I call it luck"
The next step is to maintain awareness of our cognitive biases. Overconfidence leads us to believe that we have more skill than we actually do. Hindsight bias makes us think that outcomes were "obvious" when they couldn't actually have been predicted. Confirmation bias leads us to filter data in such as way that it confirms our existing theories. Combine all those with the extraordinary complex and noise-ridden system called The Market, and we have a disastrous recipe for confusing luck with skill, one that afflicts casual investors -- particularly engineers! -- and hedge fund managers alike.
So the next time we make a decision that turns out well, insert a step before congratulating yourself (and doing it again). Ask yourself questions like, "were the odds actually in favor of this decision working out?" "How many times have I made a similar type of decision and had it fall in my favor?" "Might this be 'beginner's luck' (i.e. I happened to succeed on my first attempt, but not due to skill)?" "Am I ignoring all the times this strategy didn't work out?"
Better yet, ask someone else. No, not your supportive friend who's basically a confirmation bias factory! I'm talking about someone who tells you the things you don't want to hear. If you trust them enough to listen to what they have to say, you'll find yourself addressing problem areas that you would have otherwise ignored. (And yes, skilled financial advisors are very, very good at this!)
Datapoints in aggregate
"But wait," you say. "If good outcomes are completely random, why bother thinking through decisions at all?" Good question, and the key here is that good outcomes aren't completely uncorrelated with wise decisions -- they're just not guaranteed. The problem is that we tend to look at datapoints by themselves, or to ignore the ones that don't agree with our confirmation bias. The solution is to look at data in aggregate. How many times has this worked in my favor? (Confirmation bias would rather we just ignore the times it hasn't.) What does the research say about how often this works, not just for me, but for others who have done this? (And is that research peer-reviewed, or something I found on a forum somewhere?) If appropriate, what does the math say about the theoretical outcomes?
Remember what I said about being the casino: the object of the game is to make lots of small bets where the odds are in your favor. Overall, the Law of Large Numbers will turn those wise decisions into good outcomes. In other words, the more data points you create using wise decisions, the more likely that the overall aggregate is Good For You.
Don't rely solely on statistics
Having said that, history is littered with people -- smart people! -- who let statistics blind them to reality. You can put the odds in your favor and you can make lots of small bets, sure. But there are still "black swans" out there that can absolutely wreck you -- job loss, or sudden illness, or an accident, or a market downturn in which suddenly every asset class is nearly 100% correlated with every other (OK, that didn't strictly happen, but it was a wake-up call).
So in order to make wise decisions, we need to ask ourselves: what's Plan B? Do we have emergency savings to cover the minor disasters that come with Real Life, and/or to cover the things that can't be effectively insured, like job loss? Do we have insurance to cover the larger disasters that can be effectively insured? And do we have a retirement contingency plan that covers the scenario where the market tanks a year after we turn in our badge?
To put it another way: while part of wise decision-making is tilting the odds in our favor, another part is considering what happens when the dice fall poorly, despite our best efforts.
But what if I'm stuck?
These are all well and good, but sometimes we're faced with decisions that offer no easy framework. If you have a large sum of money to invest, then statistics would say that your best bet is to invest it all at once (because chances are higher that the market will go up than down) -- but this means making a large bet, rather than many small ones, and so the Law of Large Numbers isn't on your side. The same holds true if you're looking to diversify out of a large concentrated stock position: even if a statistical analysis indicates that diversifying immediately is your best bet, that's a large bet to make (and potentially carries a huge tax price tag.)
Sadly, there's no easy answer here.
That said, one idea is to work through the problem in the light of prospect theory: determine which outcome would be more painful, and lean towards avoiding that. For example, consider investing a large chunk of money. If you invested all of it at once and the market immediately crashed, how painful would that be -- both numerically and emotionally? Now, consider: if you decided to invest it systematically and the market had a huge run-up, such most of that chunk was still sitting in cash and missed out, how painful would that be? Which would be more painful? Choose the one that avoids that. In a very real sense, your task here is to minimize regret!