Showing posts with label Product Mgmt - AI & IOT & Robotics & Web & Apps & Backend-platform - Desktop & Mobile & TV & Devices. Show all posts
Showing posts with label Product Mgmt - AI & IOT & Robotics & Web & Apps & Backend-platform - Desktop & Mobile & TV & Devices. Show all posts

what kind of Product Manager are you - Specialist or Generalist or T-shaped

A Product Manager is like Water - It fills all gaps.

From a skillset point of view, a PM can be of following types:

1. 
A PM who is specialized in a given product domain (either payments, or search, or recommendations) is called a Specialist.

A PM who has worked extensively in a given industrial sector (EduTech, FoodTech, HealthTech, FinTech, etc.) is also categorized as one.

A PM who has the expertise to build products for a given department (marketing products, or supply chain products, or backend office products, etc.) can also be termed one.

2. 
A PM who has built products for a variety of domains/sectors/departments is a Generalist.

3. 
A T-shaped PM is a mix of above.
The base of the letter T is a vertical bar that signifies depth - This is the same as Specialization.
The top of the letter T is a horizontal bar that signifies breadth - This is the same as Generalization.


4 pillars of Product Management and their respective Archetypes

Elon Musk - SpaceX
Vision - Should be stable & not be changed for min 2-5 years

Jeff Bezos - Amazon
Strategy - Should be iterated & refined until you find product-market fit

Steve Jobs - Apple
Design - Should deliver a useful, usable, & delightful experience to your customers

Stewart Butterfield - Slack
Execution - Relentlessly doing whatever it takes to win
(Spend 60% of your time on execution - the rest 40% was for above 3 steps)

evaluating the various types of Attribution Modeling done for Marketing Analytics

Digital marketing today is scattered - People access from multiple devices, clear cookies, or use multiple browsers, which makes it difficult to track their entire journey.

To understand this, consider below example:
On Monday, a visitor sees your post on Instagram, clicks it, visits your site, and leaves.
On Wednesday, s/he clicks on your ad in IMDb.com, visits your site, and leaves.
On Friday, s/he clicks on a Google Search result, visits your site, and leaves.
On Saturday, s/he types your website's URL in a browser, visit your site, and makes a purchase.
Now, which channel (Insta, IMDb, Google Search, Direct) will you attribute your purchase to?

Attribution Modeling is a framework for analyzing which touchpoint(s) or channel(s) or interaction(s) receive the credit for a conversion.

***** Attribution Models types *****

1.
Last Interaction Attribution or Last-Click or Last-Touch

Though many of the interactions prior to the last-click are important, this model simply ignores them and gives 100% credit to the last interaction of your user before s/he converts on your site/app.

Default attribution model in most platforms, including Google Analytics.

In the case of our example, this model attributes the purchase to direct traffic.

Most accurate.

Simple & Straightforward - Easiest to evaluate.

Fit for those who have a short buying cycle.

It gives a good idea of the strongest channel.

Used if the sales funnel is wide at the top, but narrow at the bottom.

2.
First Interaction Attribution or First-Click

Exactly opposite of Last-click, it gives 100% credit to the first interaction.

In the case of our example, this model attributes the purchase to Insta.

Most accurate.

Simple & Straightforward - Easiest to evaluate.

Fit for those who have a short buying cycle.

If there is a tendency to convert customers immediately, then their first touchpoint is especially important.

Used if the business goal is bringing in new top-of-the-funnel customers.

3. 
Last Non-Direct Click

Exactly the same as Last-click, except that it eliminates any 'direct' interactions that occur right before the conversion.

In the case of our example, this model attributes the purchase to Google Search.

By eliminating the direct traffic, this model assigns value purely to the marketing strategy that led to the conversion.

4. 
Linear Attribution

Splits credit equally between all the interactions.

In the case of our example, this model attributes the purchase equally (25%) to Insta, IMDb, Google Search, Direct.

5. 
Time Decay Attribution

Exactly the same as Linear attribution, except that it also takes into consideration when the touchpoint occurred.
Interactions that occur closer to the time of purchase have more value attributed to them.
The first interaction gets less credit, while the last interaction will get the most.

Fit for those who have a long sales cycle (such as for expensive B2B purchases).

6. 
Position-based Attribution or U-shaped attribution

Exactly the same as Linear attribution, except that 40% weightage is given to 1st interaction, 40% weightage is given to last interaction, and the remaining 20% is equally split between the remaining ones.

In the case of our example, this model attributes the purchase - 40% to Insta, 10% to IMDb, 10% to Google Search, and 40% to Direct.

Fit for businesses that have multiple touchpoints prior to a conversion.

7.
Custom Attribution Models

Google Analytics lets you create these.

Obviously, gives deeper insights.

Difficult to create.

Requires a lot of data.

Used for businesses having long buying cycle and plenty of data.



Credits:
Kaushik.net/avinash/multi-channel-attribution-modeling-good-bad-ugly-models/
NeilPatel.com/blog/best-analytics-attribution-model/
Support.google.com/analytics/answer/1033861
Support.Google.com/analytics/answer/1033861
Support.google.com/analytics/answer/1665189
Support.google.com/analytics/answer/1662518
Support.google.com/analytics/answer/2909452
Support.google.com/analytics/topic/3180362
AgencyAnalytics.com/blog/marketing-attribution-models

essential Software Tools & Applications for Internet Product Managers

Project tracking & management:
JIRA

Roadmapping:
Aha!

Bug-&-issues tracking & management:
JIRA, Bugzilla, Redmine, 

Analytics:
Google Analytics, Omniture

Flowcharts & user-journeys:
Creately, LucidChart

Wireframing:
Balsamiq, Moqups, Draw.io, MS Paint, MS Powerpoint, Mockingbird, Snag-it, Pop, Hotgloo, Omnigraffle, Axure,



Mockups (extremely rarely done by PMs):
UxPin, Axure, Illustrator, Sketch, Photoshop, 

Prototypes:
Axure, Pop, Keynote, Vision, Proto.io 

Data churning:
MS Excel

Documentation:
MS Word, Grammarly, Writer Pro, Quip, iA Writer

Voice communication:
Caller, Whatsapp groups

Video communication:
Whatsapp call, Google hangout, MS Skype, MS teams

Written communication:
GMail, MS Outlook, Whatsapp groups

Data-sharing:
Googe Docs, Google Sheets, Google Drive, 

Presentations:
MS PowerPoint, Slideshare

Knowledge management & sharing:
Confluence (Atlassian), Slack, Whatsapp groups

Meeting Scheduling:
Google Calendar

Notes management:
Evernote, Recorder, Physical notebook, Full Page Screen Capture (browser extension), Just read (browser extension), Delicious (bookmark management),

Remote working:
VPN

Web conferencing:
UberConference, Adobe Connect, Cisco Webex (product)

A/B testing:
VWO

Knowledge search:
Reddit, Yahoo Answers, Google, Wikipedia, Quora, Medium, Youtube, Linkedin groups, Read out aloud (Browser extension),

Heatmapping:
CrazyEgg

Mobile testing:
Mobile/Responsive Web Design Tester (browser extension)



Credits:
Quora.com/What-are-the-essential-software-tools-and-applications-most-commonly-used-by-a-product-manager-of-web-products

mixing the magic-potion of Personalization in the drink of Entertainment

An OTT (over-the-top) media service is a streaming media service offered directly to viewers via the Internet.
OTT bypasses cable, broadcast, and satellite television platforms, the companies that traditionally act as a controller or distributor of such content.

The main goal of any OTT platform's recommender system is to get the right media in front each of its consumers.

The thumbnail of the media plays a huge role in demystifying this problem statement.

If the thumbnail can capture something compelling for the consumer, then it will act as a gateway into that media by giving him/her some visual evidence for why the media might be good for him/her.

A thumbnail may have either just the title of the media, or the media's actor's face, or a screenshot of an exciting moment from media, or screenshot of a scene from the media that conveys the essence of the same.

Understand this with the given example of the Indian 2020 biographical, period, action, drama, thriller, movie Tanhaji:

1. If a consumer has seen a few Ajay Devgan's movies recently on your platform, the thumbnail of Tanhaji-movie having his face can make the consumer click and watch it.

2. If a consumer has seen a few Saif Ali Khan Pataudi's movies recently on your platform, the thumbnail of Tanhaji-movie having her face can make the consumer click and watch it.

3. If a consumer has seen a few Indian period movies recently on your platform, the thumbnail of Tanhaji-movie, showing the actor Sharad Kelkar as the Indian king Shivaji Maharaj, can make the consumer click and watch it.

4. If a consumer has seen a few action movies recently on your platform, the thumbnail of Tanhaji-movie, showing the classic "Ajay-on-horse vs Saif-in-air" scene can make the consumer click and watch it.

So, now the point/problem is:
How to find the thumbnail that will be compelling enough for the consumer to click?

One solution to this problem is:
Find the single perfect thumbnail using AB, for each media.

A better solution would be:
Find the best thumbnail for each consumer that highlights the aspects of a given media that are specifically relevant to them.

Challenges in showing a different thumbnail to each different customer:

1.
Because only a single thumbnail can be used to represent each media in each place for a given user, the thumbnail selection becomes a chicken-and-egg problem operating in a closed-loop: if a user plays a media it can only come from the thumbnail that we decided to present to that user.
So, the challenge is to understand that when the user played a media, whether he was influenced by the thumbnail and when the user played it regardless of which thumbnail was presented.

2.
The next challenge is to understand the impact of changing thumbnail for a given media between sessions - but continuous changes can also confuse people.
Does changing thumbnail reduce the findability of the media and make it difficult to visually locate the media again?
Does changing the thumbnail itself lead the user to now consider it, which s/he had previously rejected?
Changing thumbnails also lead to an attribution problem, as it becomes unclear which thumbnail made a user in a given media.

3.
The next challenge is of understanding how a given thumbnail performs in relation to other thumbnails we put up the same page or session.
Maybe a bold close-up of the main character works for a media on a page because it stands out compared to the other thumbnail.
But if every media had a similar thumbnail then the page as a whole may not seem as compelling.
Looking at each thumbnail in isolation will not be enough and we need to think about how to select a diverse set of thumbnails across various medias on a page and across a session.
The effectiveness of a thumbnail for a media will also depend on what other types of evidence and assets (e.g. synopses, trailers, etc.) we also display for that media.

4.
To achieve effective personalization, we will need a good pool of thumbnails for each media - By good, we mean engaging, informative and representative of a media to avoid “clickbait”.
The set of thumbnails for a media also needs to be diverse enough to cover a wide potential audience interested in different aspects of the content.

5.
Finally, there are engineering challenges to personalize thumbnails at scale.
When you have a platform having millions of media, and hence millions of thumbnails, using personalized selection for each media means handling a peak of over million requests per second with low latency.
Such a system must be robust: failing to properly render the thumbnail in UI would significantly degrade the experience.
Also, the personalization algorithm also needs to respond quickly when a new media goes LIVE on the platform, which means rapidly learning to personalize in a cold-start situation.
Then, after launch, the algorithm must continuously adapt as the effectiveness of thumbnail may change over time as both the media evolves through its life cycle and member tastes evolve.

Credits:
Microsoft.com/en-us/research/wp-content/uploads/2016/02/p661.pdf
Netflix.com/in/
En.Wikipedia.org/wiki/Over-the-top_media_service
NetflixTechBlog.com/selecting-the-best-artwork-for-videos-through-a-b-testing-f6155c4595f6
En.Wikipedia.org/wiki/Recommender_system
Media.Netflix.com/en/company-blog/the-power-of-a-picture
NNGroup.com/articles/personalization/
NetflixTechBlog.com/artwork-personalization-c589f074ad76
NNGroup.com/articles/customization-personalization/

Rdio - Music Streaming Product - 2015 Growth Failure story

In this post we will study "Rdio", which was a popular Music Streaming Product that failed miserably.

******* Choronology *******

2010 Aug
Rdio; the first modern music streaming service; was launched by Skype's founders Niklas Zennström and Janus Friis.
It had to compete with services such as Deezer, MOG, Napster, Rhapsody, and Spotify.
It offered a $5 web-only streaming plan (on the assumption you might not have a mobile device) and a BlackBerry app (in case you had a bad one).
Its catalog was limited to 7M songs, well short of the 30M tracks that it and its rivals now provide.


2013 Sep
Rdio added a music recommendations feature that delivered personalized albums, stations, and playlists.

2014 Jan
Rdio introduced some free streaming options, supported by audio advertisements.

2015 Nov
Rdio filed for bankruptcy.
Reached a deal to sell certain assets and intellectual property to a competitor, Pandora, for $75M.

2015 Dec
Rdio service was discontinued.

******* Experience *******

Using Rdio felt like the future.

Though, securing label deals took so long that the app was in development for two years before it launched, and it showed in the polished product delivered by its team.

Its blue-and-white design was calming.

Its simple grid of album artwork was a powerful rebuttal to iTunes’ nightmare spreadsheets.

It had innovative social features, showing you what your friends were streaming in real time.

It had a "heavy rotation" playlist that highlighted albums based on how many friends had listened to them.

"Social from the ground up — it sounds like marketing speak, but it was legit," said Chris Becherer, Rdio’s head of product. "The founding premise was the best music recommendations come from the people you know. That was the whole idea."

It got music people to explore and listen to more new+old music than they ever had before.
It always surfaced things people didn’t yet know they should be listening to.

Overall an excellent product.

******* Growth & Marketing *******

Early as it was to the United States, Rdio was born in the shadow of Spotify, a cunning and well-financed competitor that excelled at generating buzz — and using that buzz to acquire paid subscribers.

As streaming music became a playground for giants, Rdio turned to a terrestrial radio company in a last-ditch effort to grow the user base.

Ultimately, executives decided that Rdio’s only future lay in becoming part of an internet-based platform, even if it meant disassembling the service they had been building for more than five years.

Even in late 2010, when it began to spread among design-savvy early adopters in San Francisco, people were already talking about the coming launch of Spotify.
The Swedish streaming service wouldn’t launch in America for several months, but it quickly came to define online music in the popular imagination.
Its secret: free on-demand streaming, supported by advertising.
By contrast, Rdio required a paid subscription.

Rdio eventually developed a free tier, but it came long after Spotify launched in the United States.
More pressingly, the company struggled to make the case for its own unique service.

Rdio never had a dedicated marketing chief for more than a few months at a time.
Early on, the company contracted with West, a San Francisco-based agency run by Allison Johnson, Apple’s former head of marketing.
But many people inside the company blamed the lack of in-house marketers on its lack of traction. Later, Mark Ruxin, who joined Rdio after it acquired his app Tastemaker, served in the role.
But he only served in it for a few months before leaving.

By 2013, Spotify had rocketed to 24 million users, 6 million of whom paid.

Struggling to stand out, Rdio turned to Cumulus Media, which operated 525 terrestrial radio stations.

Cumulus took a large equity stake in the company; in return, its sales force began selling ads for Rdio, which enabled the company to finally offer the free, ad-supported version of the service that Spotify had been offering in various forms since 2008.

Cumulus also promised to promote Rdio on its popular stations.

But the resulting signups were apparently nothing to brag about.

******* Feature Prioritization *******

Rdio sometimes focused on the wrong things.

It invested many product cycles in refining its queue — a place to collect things you want to listen to later.

Every other music streaming service offers a queue that’s a simple list of tracks.

But if you dragged an album or a playlist into Rdio’s queue, Rdio would recognize it as a distinct object, so you could drag and drop an album above a track, or a full playlist below an album.

"That was not a major differentiating factor," says Wilson Miner (Design Head) "If we hadn’t had something like that, nobody would have noticed and it would have been fine. I still wish we could have solved it, but it was more of a personal quest than a brutally honest assessment of priorities."

******* Leadership & Radio-focus *******

Jun 2013, just before the Cumulus deal was announced, Rdio CEO Drew Larner announced he was stepping down.

Former employees say Larner was never totally comfortable being CEO; his expertise was in dealmaking, and he thought the company needed a leader whose talents lay in product and user growth.

Malthe Sigurdsson, its head of product and the person who oversaw its innovative design efforts, quit a month after Larner.

In Nov, Rdio laid off a third of its staff.

And while development on the core product continued, it increasingly felt radio-focused.

In August it added live radio stations to the app, a move designed to capitalize on the success of iHeartRadio, an app developed by the company formerly known as Clear Channel.

Earlier this month, Cumulus wrote down its investment in Rdio by $19 million.

******* Economics of Music *******

The economics of streaming music are brutal. Record labels have nearly all the leverage, and take most of the gross revenue from streaming services.

The only way to win is to achieve a massive scale — which is why Spotify has raised more than $1 billion, spending heavily to add subscribers in hopes they will lead to a sustainable business.

Rdio realized this only belatedly.

"Rdio, I guess, made the mistake of trying to be sustainable too early," Wilson Miner says. "That classic startup mistake of worrying about being profitable and having a business that makes any sense before you’ve reached this astronomical growth curve. Which is partly the trap of the business model itself. Because of the content licensing deals, the margins for the business were so incredibly thin. No matter what we did, the labels made the lion’s share of the revenue. You have to make it up with extreme volume, which is why you see Spotify going after every human being in the world."

And yet even with more than 75 million users and 20 million paid users, Spotify still isn’t profitable. It remains to be seen whether Apple or Google can turn their own streaming offerings into viable businesses — or whether they will simply use music as a loss leader to draw consumers further into their respective ecosystems, making the money back on hardware sales or other services.

******* Cause of Death *******

Multiple!
Marketing failure, Growth failure, and Leadership failure.

Niklas Zennström


Credits:
Linkedin.com/in/drew-larner-5b51bb34/
Linkedin.com/in/wilsonminer/
Linkedin.com/in/becherer/
Linkedin.com/in/malthe/
Linkedin.com/in/niklaszennstrom/
En.Wikipedia.org/wiki/Rdio
TheVerge.com/2015/11/17/9750890/rdio-shutdown-pandora

Wesabe - the 2010 FinTech Product Failure story

Wesabe was a personal finance company.

It analyzed a user's financial data to provide appropriate advice on how to save money.

Established in Dec 2005, it's site went live in Nov 2006, and they shut down in Jul 2010.

Received an approx $4M funding from Union Square Ventures and O'Reilly Alphatech Ventures.
Wesabe started generating revenue in late 2008, ran completely out of invested funds and survived solely on revenue almost 9 months before closing the company.

Marc Hedlund was the first person to start work on Wesabe and formally co-founded the company (as Chief Product Officer) with his high school friend, Jason Knight (CEO). 
Jason later left due to a family illness, Marc took over as CEO; without a formal peer; for the final two years.

When the company closed, Marc wrote a postmortem on the experience.
Following is a paraphrased; & improvised with images; version of his blogpost.

******** HISTORY ********

In Nov 2006, Wesabe launched as a site to help people manage their personal finances.

We certainly weren’t the first to try to tackle this problem through a web app, but we were the first of a new wave of companies that came out in the months that followed, characterized but what some would call a Web 2.0 approach to the problem.

Like Flickr and del.icio.us, we relied on community and features such as tags; unlike some of the previous attempts, we tried to automatically aggregate and store all of our users’ financial accounts on the web (instead of relying on manual data entry, say, or desktop storage of the data); and most especially, we tried to learn from the accumulated data our users uploaded, and make recommendations for better financial decisions based on that data.

If every copy of Quicken started out as a blank spreadsheet, Wesabe tried to accumulate knowledge from users and data that would fill in some of that spreadsheet for you, and point you in the direction of better choices.

Wesabe's website

Even before we launched, we heard about other people working on similar ideas, and a slew of companies soon launched in our wake.

None of them really seemed to get very far, though, and we were considered the leader in online personal finance until September 2007, when Mint launched at, and won, the first TechCrunch 40 conference.

From that point forward we were considered in second place at best, and they overshadowed our site and everyone else’s, too.

Two years later, Mint was acquired by Intuit, makers of Quicken (and after Mint’s launch, the makers of Quicken Online) for $170M.

Mint's website

I made nearly all of the product decisions myself, and was notorious with Jason and our board as being very hard-headed about those decisions.

While I relied on many other people in making product choices, I also hired and managed all of those people, so that group was at the least a reflection of who I thought had the right values and ideas.

******** FACTS ********

Wesabe launched about 10 months before Mint, but we didn’t capitalize on that early lead.

There’s a lot to be said for not rushing to market, and learning from the mistakes the first entrants make.
Shipping a MVP immediately and learning from the market directly makes good sense to me, but engaging with and supporting users is anything but free.

Observation can be cheaper.
Mint did well by seeing where we screwed up, and waiting to launch until they had a better approach.

Mint’s design was exceptional, but if other, stronger forms of lock-in are in place first, design alone can’t win a market, nor can it keep a market.

Neither of Mint nor Wesabe, bore any resemblance to a typical Silicon Valley success story, with traffic surging up and to the right (YouTube, Twitter, etc).
Mint aggressively acquired users by paying for search engine marketing (reportedly spending over $1 for each user), while Wesabe spent almost nothing on marketing; yet in the end we grew at about 1/5th the rate they did.
Their traffic dropped substantially for the 6 months after their acquisition, and has had sawtooth traffic after that.
Our patterns followed non-scalable curves (influenced primarily by press wins, economic conditions, and sometimes drafting on Mint’s coverage).

******** POSTMORTEM - A ********

We chose not to work with Yodlee, but failed to find or make a replacement for them (until too late).

Yodlee is a company that provides automatic financial data aggregation as a web service.
They screen-scrape bank web sites (that is, read the payee and amount and date by parsing them out of the bank’s web site, writing a custom parser for each bank they support).

When we talked to Yodlee in 2006, the company was crumbling, having failed to get acquired and losing executives.

They were also very aggressive in negotiation, telling us they would give us six months’ service nearly free and then tell us the final price we’d be charged going forward.

Since they had effectively no competitors, we didn’t believe we should tie our company to a single-source provider, especially one in very bad business shape.

Mint used Yodlee (at least until they were acquired - I’m not sure what they’re doing now) to automatically get user’s data from bank sites and import them into Mint, and as a result had a much easier user experience getting bank data imported.

Wesabe built our own data acquisition system, first using downloadable client programs (partially because that was easier and partially to preserve users’ privacy) and later using a Yodlee-like web interface, but the Yodlee-like version didn’t launch until six months after Mint went live, and even then didn’t really work as a near-complete replacement for some time after.

A good friend argues that our mistake was not using Yodlee in the first place, and maybe – probably – it was.

I believe, though, that we could have made that choice as long as we immediately assumed that someone else would eventually sign up with Yodlee, and that we had to be at least as good if not better than what Yodlee provided, however we got there.

PageOnce, for instance, has not used Yodlee, but has grown very significantly in the same time, using a combination of other aggregation methods that were more effective than ours.

Mint’s dependence on Yodlee apparently suppressed their acquisition interest among companies that knew Yodlee well (such as Microsoft, Yahoo, and Google); since we had developed our own technology for aggregation, we didn’t have that particular problem, and in fact had some acquisition interest simply for the aggregator we’d built.

We just didn’t build it nearly fast enough. 

******** POSTMORTEM - B ********

Mint focused on making the user do almost no work at all, by automatically editing and categorizing their data, reducing the number of fields in their signup form, and giving them immediate gratification as soon as they possibly could.
We completely sucked at all of that. 

Instead, I prioritized trying to build tools that would eventually help people change their financial behavior for the better, which I believed required people to more closely work with and understand their data.
My goals may have been noble, but in the end we didn’t help the people I wanted to since the product failed.
I was focused on trying to make the usability of editing data as easy and functional as it could be.

Mint was focused on making it so you never had to do that at all.
Their approach completely kicked our approach’s ass.

But their data accuracy; how well they automatically edited; was really low.
And anyone who looked deeply into their data at Mint, especially in the beginning, was shocked at how inaccurate it was.
The point, though, is hardly anyone seems to have looked.

Between the worse data aggregation method and the much higher amount of work Wesabe made you do, it was far easier to have a good experience on Mint, and that good experience came far more quickly.

******** CRUX ********

1.
That one mistake (not using or replacing Yodlee before Mint had a chance to launch on Yodlee) was probably enough to kill Wesabe alone.

2.
Everything I’ve mentioned – not being dependent on a single source provider, preserving users’ privacy, helping users actually make positive change in their financial lives – all of those things are great, rational reasons to pursue what we pursued. 
But none of them matter if the product is harder to use, since most people simply won’t care enough or get enough benefit from long-term features if a shorter-term alternative is available.


3.
A domain name doesn’t win you a market.
Launching second or fifth or tenth doesn’t lose you a market.
You can’t blame your competitors or your board or the lack of or excess of investment.

4.
Focus on what really matters: Making users happy with your product as quickly as you can, and helping them as much as you can after that. 
If you do those better than anyone else out there you’ll win.
I think in this case, Mint totally won at the first (making users happy quickly), and we both totally failed at the second (actually helping people).
No one, in my view, solved the financial problems of consumers - No one even got close.
Yes, both products helped some people – ours mostly through a supportive community and theirs mostly through giving people a rough picture of where their money has gone.

5.
But when we analyzed the benefits we saw for our users, and when Mint boasted about the benefits they saw for their users, the debt reduction and savings increase numbers directly matched the national averages.
Because our products existed during a deep financial crisis, consumers everywhere cut back, saved more, and tried to reduce their debt.
Neither product had any significant impact beyond what the overall economy led people to do anyways.

6.
Changing people’s behavior is really hard. 
No one in this market succeeded at doing so – there is no Google nor Amazon of personal finance. 

Marc Hedlund


Credits:
Linkedin.com/in/precipice/
En.Wikipedia.org/wiki/Wesabe
Blog.Precipice.org/why-wesabe-lost-to-mint/
TheSmarterWallet.com/2009/wesabe-review-free-online-money-management-tool/

if you Choose an Answer to this Question, what is the Chance that you will be Correct

We will try to solve the famous mathematical probability paradox in this post, which goes like this:

if you choose an answer to this question at random,
what is the chance you will be correct?
(A) 25% 
(B) 50% 
(C) 60% 
(D) 25%
******** STEP 1 ********

We will have to assume that this is a multiple choice question (MCQ), as there are 2 options - A & D - with the same answer (25%)

So, we have 4 options.

Following are the logic formulas that we will use to solve the problem:

Logic S
If only 1 option of the 4 options is correct, then the probability would be =
1/4 =  25%

Logic K
If 2 options of the 4 options are correct, then the probability would be =
2/4 =  50%

Logic G
If 3 options of the 4 options are correct, then the probability would be =
3/4 =  75%

Logic Z
If only all 4 options are correct, then the probability would be =
4/4 =  100%

Logic V
If none of 4 options are correct, then the probability would be =
0/4 =  0%

******** STEP 2 ********

As we can see in the logic S/K/G/Z, 
a 60% probability of getting the right answer does not exist.
So, we rule out option C

If option A is correct, then option D will also be correct.
Or to say, 2 options are correct.
So, going by logic K:
if 2 options of the 4 options are correct, then the probability would be =
2/4 =  50%
But the value of A & D is 25%
So, we rule out both options A & D

Finally, if option B is correct.
Or to say, 1 options is correct.
So, going by logic S:
if only 1 option of the 4 options is correct, then the probability would be =
1/4 =  25%
But the value of B is 50%
So, we rule out option B

******** STEP 3 ********

Summary/Answer is hence, that there is no correct answer.
Or to say, 0 option is correct.
So, going by logic V:
if none of the 4 options is correct, then the probability would be =
0/4 =  0%
Which is unfortunately not listed here!
And, that's the reason that this questions is a paradox.

Apple's Job Interview Questions

We have a cup of hot coffee and a small cold milk out of the fridge.
The room temperature is in between these two.
When should we add milk to coffee to get the coolest combination earliest (at the beginning, in the middle, or at the end)?

How much does the Empire State Building weigh?

How do you check if a binary tree is a mirror image on left and right sub-trees?

What superhero would you be and why?

Explain what RAM is to a five year old.

How does an airplane wing work?

Draw the inside architecture of an iPhone.

Give me 5 ways of measuring how much gasoline is in a car.

If you have 2 eggs, and you want to figure out what's the highest floor from which you can drop the egg without breaking it, how would you do it?
What's the optimal solution?

How would you break down the cost of this pen?

Describe an interesting problem and how you solved it.

Explain to an 8-year-old what a modem/router is and its functions.

How many children are born every day?

You have 100 coins laying flat on a table, each with a head side and a tail side.
10 of them are heads up, 90 are tails up.
You can't feel, see or in any other way find out which side is up.
Split the coins into two piles such that there are the same number of heads in each pile.

How would you test your favorite app?

There are three boxes, one contains only apples, one contains only oranges, and one contains both apples and oranges.
The boxes have been incorrectly labeled such that no label identifies the actual contents of the box it labels.
Opening just one box, and without looking in the box, you take out one piece of fruit.
By looking at the fruit, how can you immediately label all of the boxes correctly?

Scenario:
You're dealing with an angry customer who was waiting for help for the past 20 minutes and is causing a commotion.
She claims that she'll just walk over to Best Buy or the Microsoft Store to get the computer she wants. Resolve this issue.

A man calls in and has an older computer that is essentially a brick.
What do you do?

Are you smart?

What are your failures, and how have you learned from them?

Have you ever disagreed with a manager's decision, and how did you approach the disagreement?
Give a specific example and explain how you rectified this disagreement, what the final outcome was, and how that individual would describe you today.

You put a glass of water on a record turntable and begin slowly increasing the speed.
What happens first — does the glass slide off, tip over, or does the water splash out?

Tell me something that you have done in your life which you are particularly proud of.

Are you creative?
What's something creative that you can think of?

Describe a humbling experience.

What's more important, fixing the customer's problem or creating a good customer experience?

Why did Apple change its name from Apple Computers Incorporated to Apple Inc.?

You seem pretty positive.
What types of things bring you down?

Show me (role play) how you would show a customer you're willing to help them by only using your voice.

What brings you here today?

Given an iTunes type of app that pulls down lots of images that get stale over time, what strategy would you use to flush disused images over time?

If you're given a jar with a mix of fair and unfair coins, and you pull one out and flip it 3 times, and get the specific sequence heads heads tails, what are the chances that you pulled out a fair or an unfair coin?

What was your best day in the last 4 years?
What was your worst?

When you walk in the Apple Store as a customer, what do you notice about the store/how do you feel when you first walk in?

Why do you want to join Apple and what will you miss at your current work if Apple hired you?

How would you test a toaster?

If you are to go up the mountain one day and come down the next day, leaving at the same time, will you ever be at the same place at the same time of day?

Are you the type of person people come to for technical issues?

62 - 63 = 1
Changing only one element (either digit or operand), make this statement true.

What is your favorite ice cream?

How do you create an efficient supply chain model?

Describe a time when you hurt a friend and how did you handle it?

How would you plan a hand gliding trip for your colleagues to North Korea?

What sort of data would you ask for when looking to bring iPhone to a new market?



Source:
https://www.businessinsider.in/44-of-the-hardest-questions-Apple-will-ask-in-a-job-interview/What-sort-of-data-would-you-ask-for-when-looking-to-bring-iPhone-to-a-new-market-Finance-candidate-at-Apple/slideshow/58550053.cms

2020 trends - EdTech (Education Technology)

1.
AI, RPA, Chatbots to replace administrative tasks

2.
Adaptive Learning Systems use Analytics, DataScience, ML/AI for content recommendation & customized learning path creation (evaluate a student’s competencies, find weak areas, and then present supporting materials).

3.
Using Analytics, DataScience, ML/AI to predict student behavior, and to predict student success and recruitment.

3.
AI/ML to automate the creation of course summaries & flashcards

4.
Smartboards


5.
Voice Recognition & NLP

6.
Use of Augmented Reality (AR) and Virtual Reality (VR) gives students 1st hand experience - For example, the students can perform virtual surgeries rather than watching a video or live demonstration.


7.
Text to Speech technologies can be used by students to read out books to them.


8.
AI/ML can be used to do auto-translation of books that are otherwise not available in a given language.

9.
The Blockchain aka Distributed Ledger Technology (DLT)'s advantage is that it’s a decentralized & transparent way to transact data - Those characteristics make it a fraud-proof way to transact and authenticate information. Schools will use blockchain for cost-effective cloud storage options and for securing student record transfers. Online educational platforms that still struggle with accreditation (verify skills and knowledge) will find its solution in DLT - Massive Open Online Courses (MOOCs) can use tools like open badges to solve the problem of authentication, scale, and cost - Open badge platforms like Mozilla’s will help job seekers and student applicants collect and publicly display their accomplishments.

Credits:
ELearningIndustry.com/2019-edtech-trends-excited
Youtube.com/watch?v=0U05WeXPGlk
Youtube.com/watch?v=-m6wHd_WENM
Youtube.com/watch?v=etn2zCa7n40

2020 trends - FinTech (Financial Technology)

1.
Hyper-personalization via big data and AI
For many years, marketing experts espoused the benefits of personalization to attract customers and keep them loyal. Today, thanks to big data and artificial intelligence that helps us process, store, and drive insights from the data, hyper-personalization is possible on an unprecedented scale. Financial institutions now have information about their customers' behavior and social and browsing history. AI facilitates real-time omnichannel integration of these insights to deliver a personalized one-to-one marketing experience for their customers at the time when the information is most relevant and useful.

2.
Robotic process automation (RPA)
During 2020, robotic process automation (RPA) will continue to impact financial institutions to help them be more efficient and effective as well as help ensure they meet federal and state compliance requirements. Today’s advanced RPAs don’t have to be explicitly programmed to perform tasks; they can simply observe what humans do and then automate or suggest improvements to processes. This includes processes such as customer onboarding, verification, risk assessments, security checks, data analysis and reporting, compliance processes as well as most other repetitive administrative activities.

3.
Conversational interfaces
According to Gartner, by 2020, chatbots will interact with the customers of 85% of banks and businesses. By eliminating human involvement in these interchanges, productivity, and speed improve. In fact, according to one report, financial chatbots save over four minutes on every interaction. This is a booming area due to the progress made in natural language processing and speech generation. Customers of financial institutions have come to rely on conversational interfaces to provide 24/7 service, instant responses to queries, and quick complaint resolution to improve personal banking significantly. Conversational interfaces also provide an easy and economical way for organizations in the financial sector to receive customer feedback.

4.
Blockchain
Blockchain, a special immutable computer file that is decentralized and distributed, is disrupting financial institutions. Blockchain can make things more efficient in the financial services industry. Since fraud and identity theft cost financial institutions billions of dollars annually, blockchain has the potential to save the industry from experiencing these significant losses. Blockchain in fintech is expected to reach $6,700 million by 2023 in the United States. Financial institutions will use blockchain for smart contracts, digital payments, identity management, and trading shares.

5.
Mobile payment innovations
One of the latest “big things” in fintech is the growth of the mobile payments industry. Consumers want payments to be instant, invisible, and free (IIF). Mobile payment innovations might even do away with our traditional wallets as global consumers are less reliant on cash. Google, Apple, Tencent, and Alibaba already have their own payment platforms and continue to roll out new features such as biometric access control, inducing fingerprint, and face recognition. One of the most popular payment methods in China and used by hundreds of millions of users every day is WeChat Pay. Alibaba’s Alipay, a third-party online and mobile payment platform, is now the world’s largest mobile payment platform. Many mobile payment platforms are building programs and offers based on the user’s purchase history.

While many financial institutions are continuing to adopt new technology to enhance operations and improve customer service, these five trends will provide exciting avenues for innovation. Financial institutions realize they must learn how to use fintech to their competitive advantage.

6.
Crowdfunding platforms
 allow internet and app users to send or receive money from others on the platform and have allowed individuals or businesses to pool funding from a variety of sources all in the same place. Instead of having to go to a traditional bank for a loan, it is now possible to go straight to investors for support of a project or company. And while their applications range from family and friends funding to fan and patron funding, the number of crowdfunding platforms have multiplied over the years.

7.
Insurtech
Fintech has even disrupted the insurance industry. In fact, insurtech (as it's been so-called) has come to include everything from car insurance to home insurance and data protection.

8.
Robo-Advising and Stock-Trading Apps
Robo-advising has disrupted the asset management sector by providing algorithm-based asset recommendations and portfolio management that have increased efficiency and lowered costs. Since the rise of more advanced technology that can analyze various portfolio options 24/7, financial institutions have adapted to offer online robo-advising services - including the likes of Charles Schwab (SCHW) - Get Report and Vanguard. Additionally, other popular robo-advising services include Betterment and Ellevest. Perhaps one of the more popular and big innovations in the fintech space has been the development of stock-trading apps. When once investors had to go directly to a stock exchange like the NYSE or Nasdaq, now, investors can buy and sell stocks at the tap of a finger on their mobile device. And with inexpensive and low-minimum apps like Robinhood or Acorns, investing from anywhere with any budget has never been easier.

9.
Budgeting Apps
One of the most common uses of fintech in 2019 is budgeting apps for consumers, which have grown exponentially in popularity over the years. Before, consumers had to create their own budgets, gather checks, or navigate excel spreadsheets to keep track of their finances. But after the fintech revolution prompted the development of financial services apps, consumers can easily and efficiently keep track of their income, expenses and other budgeting tools that have revolutionized the way consumers think about their money. Budgeting apps like Intuit's (INTU) - Get Report Mint help consumers track their income, monthly payments, expenditures and more - all on their mobile device.



Credits:
Forbes.com/sites/bernardmarr/2020/12/30/the-top-5-fintech-trends-everyone-should-be-watching-in-2020/#5d15555d4846
TheStreet.com/technology/what-is-fintech-14885154
PaymentsJournal.com/fintech-trends-everyone-should-look-for-in-2020/

Pricing a Good which goes from a Manufacturer to Wholesaler to Retailer to Customer (class 2 of 2)

There are a lot of factors that affect the decision of price-pointing any given product. We studied some of the major these factors in the 1st part of the story here:
https://saurabhkautilyagupta.blogspot.com/2020/03/pricing-factors-market-competition-quality-cost-supply-chain-hops-profit-margin-product-management.html

After collecting the above data points, let us start pricing calculation...

******************

From the Manufacturer's perspective:

Manufacturing cost = $100

Innterests & taxes = $50

Supply chain cost (warehousing, packing, shipping) = $20

Assuming we want to keep a net profit margin of 10%, we should price it at =
Cost + 10% margin.

Cost = Sum total of all costs =
($100 + $50 + $20) = $170

So, we would price our product at =
$170 + (10% of $170) = $170 + $17 = $187

So, the price at which the Manufacturer will sell the product to the Wholesaler =
$187

******************

From the Wholesaler's perspective:

Cost of Good = $187

Interests & taxes = $3

Marketing cost = $5 (initially - this will/might go down as sales increase)

Supply chain cost (warehousing, packing, shipping) = $5

Totals costs = $200

Assuming s/he wants to keep a net profit margin of 5%, we should price it at =
Cost + 5% margin =
$200 + (5% of $200) = $220 + $10 = $210

So, the price at which the Wholesaler will sell the product to the Retailer =
$210

******************

From the Retailer's perspective:

Cost of Good = $210

Interests & taxes = $5

Supply chain cost (warehousing, packing, shipping) = $5

Totals costs = $220

Assuming s/he wants to keep a net profit margin of 5%, we should price it at =
Cost + 5% margin =
$220 + (5% of $220) = $220 + $11 = $231

So, the price at which the Retailer will sell the product to the Customer =
$231

******************

Let us assume, that the competitors' product's price = $250

So, the MRP of our product can be kept at = $240

At $240, we are cheaper and better (assuming we are offering better features) than the competition, and the wholesalers & retailers are also making a decent margin. The retailers can offer the remaining $9 (=$240-$231) to the customers as a discount.

******************

Additional pointers:

If the product is also expected to generate revenue (this logic is not valid for all physical products, but for a product like Kindle which is not just a product - it is also a platform for selling more products), the Manufacturer can decide to reduce its initial Margin by $10.

After launching the product, we can change the prices multiple times (in the name of discounts, flash-sales, promotions, offers, etc) - Lower the prices if expected sales do not happen (or if customers give us feedback/suggestions about lowering the pricing) & increase the prices if the sales is increasing - Take note of the demand at each price point and decide the best price-point of our product.

a sample Sales-ad by Jabong (now part of Walmart group)

Pricing a Good which goes from a Manufacturer to Wholesaler to Retailer to Customer (class 1 of 2)

There are a lot of factors that affect the decision of price-pointing any given product.
Some of these factors are:

Pricing of similar products in the market by competitors?

Pricing of Similar products in the market by us?

Are we okay with launching a product at a drastically-different (high/low) pricing than our other products?

What is our usual product pricing & profit margin% for all other products that we have so far?

All the costs that went into manufacturing, marketing, etc.?
Ideally here we will include the operating expenses (like packaging costs, supply chain costs, etc.), the interests we pay on liabilities, and the taxes we pay to the government.

Length of the distribution chain - How many hops does it take before the product reaches in the hand of our customer?

Net Profit Margin% that we want to make?

Are we willing to take a hit in profit?

Are we willing to sell it at loss, for some time, to capture the market?

Is our product better than the competitors' products?

Who is our TG?

What price-ranges does our target audience usually buy?

What our customers perceive of us?

What kind of brand are we - new & popular, new but not-popular, old & popular, old but not-popular?

Do we plan to launch this product under our existing brand-name or with a new brand-name?

Where will we be selling it - on own website, on other online stores, on offline stores?
If we sell offline, a price multiplier will have to be added to make sure that all the people - including wholesalers & all the retailers in the chain can make a profit by selling our product.

Is it a necessity or a luxury or a premium product?

Is there such a stiff competition that the final price is already decided?

Is there a scarcity of this product in the market?

After you collect the above data points, you will be able to start pricing calculation - that we have done in the 2nd part of this story here:
https://saurabhkautilyagupta.blogspot.com/2020/03/pricing-cost-expense-tax-gross-net-profit-margin-markup-product-management.html

unleashing the power of Google Analytics' Cohort

Google defines 'COHORT' as a group of users who share a common characteristic; like 'Acquisition date'; identified by an Analytics dimension. A cohort analysis, hence, is the process of analyzing this behavior of groups of users.

sample Cohort-analysis report

Cohort Type:
GA, only gives the option of Acquisition date, as of now.
This is what goes on the vertical axis.

Cohort Size:
Days, weeks or months

Metric:
This is the metric that is being measured for each cohort.
This could be User retention, Revenue, Session duration, Pageviews, Goal Completions per user, Pageviews per user, Session Duration per user, Revenue per user, Sessions per user, Transactions per user etc.

Date range:
This is the range of data that you want to analyse.

*************************

Cohort report:

the Vertical section:
Shows the count of users, grouped as per their acquisition dates (cohort-type), grouped into weeks (cohort sizes), for last 6 weeks (date range)

the Horizontal section:
Shows last 6 weeks (date range)

the colorful Matrix:
Each row of the matrix shows how did the users acquired (visited your site/app for 1st time) in a given week, behaved (Mertic's behavior) in the coming 6 weeks.
So, in the image above, because we chose to analyze the User Retention metric, in the 1st row, for the visitors acquired in the "26 Feb to 4 Mar" week:
Week 0 is; obviously/always/for-all; 100%
Week 1 is 3.71%, which means that 3.71% of the visitors acquired in the "26 Feb to 4 Mar", got retained ie. came back the following week.


Credits:
Medium.com/the-data-dynasty/the-single-most-overlooked-report-in-google-analytics-yet-most-powerful-6ec90eba243a
Neilpatel.com/blog/cohort-analysis-google-analytics/
Youtube.com/watch?v=N02uDh-7Kcg

demystifying the Mathematics behind Multivariate Experiments

Imagine your site/app to be a bag full of balls of two colors - red & black - in unequal proportions.

Because you can't look inside the box to count the number of balls of each color, you ask a couple of your friends to pick one ball each.

The reason for doing this activity is that, by checking a sample of balls, you want to estimate the entire count.

This is the basis of Multivariate Testing/Experiments as well:

Your site/app has a button whose existing color = red.
While the color you think the button should be = black.

Around 100k people visit your site daily.

On average, 3% of daily visitors have been clicking the red button, for the last 30 days.
Technically speaking, the red button's CTR = 3%.
Note that "3%" is an average, which means that it is the MEAN of daily-CTRs of the last 30 days (i.e. there would have been days when the CTR would have been 9% and other days when the CTR would have been 0% as well).

Now, you want to test how the black color button will perform.
Though you are confident that that the black button will get a better CTR, you can not just replace the red button with the black button and show it to all users, as there is a chance that users might not like it at all, and hence not click it at all.

So, you let a couple of your visitors view the black color button.

The reason for doing this activity is; just like the story that I told at the start of this post; that, by checking a sample, you want to estimate the entire click-count.

Now, let's say you showed the black button to 1k (1% of total) visitors for 30 days, and the following are the results that you get:


We can see from the above image that on 1st & 2nd day, the CTR of the black button was 0 (no one clicked), while on 26th, 27th, 28th, 29th, 30th day it was 0.09 (9 out of each 100 visitors who saw the button clicked on it).

Also, at the bottom of the 4th column, we have calculated the Mean of the CTR's data = 5


Also, in the 5th column, we have calculated the Square of (CTR data - CTR Mean) for each day.

And, at the bottom of the 5th column, we have calculated Sum of Square of (CTR data - CTR Mean) 0.026, which we will now use to calculate the Standard Deviation (sometimes, also called the Standard Error), whose formula is:


Standard Deviation
= Square-Root of [{Sum of Square of (CTR data - CTR Mean)} / {Impressions total count}]
Square-Root of [{0.026} / {30000}]
= 0.0009 or 0.09%

*** So, the Mean of CTR data is 5% with a Standard Deviation of 0.09% ***

Now, we will apply the 68-95-99.7 rule of Statistics, assuming the data is normally distributed (the rule is depicted in the image below):

Rule 1
68% of the data falls within ONE standard deviation (=0.09 in this case) of the mean.

So, 68% of our CTR-data would be between 4.91% (=5%-0.09%*1) and 5.09% (=5%+0.09%*1)

Rule 2
95% of the data falls within TWO standard deviations of the mean.
Actually, precisely, it is not TWO, but 1.96

So, 95% of our CTR-data would be between (5%-[0.09%*1.96]) and (5%+[0.09%*1.96])

So, 95% of our CTR-data would be between 4.82% and 5.18%

Rule 3
99.7% of the data falls within THREE standard deviations of the mean.


So, the final result of the experiment will be:
1. We are 68% confident that the CTR of the black button is between 4.91% to 5.09%
2. We are 95% confident that the CTR of the black button is between 4.81% to 5.18%


the '68-95-99.7' Rule

============================

Notes/Tips:

1.
These exact coefficients; for example, 1.96; are called the Standard Error or Z-score; denoted by 'Z'; of Standard Deviation - It can be calculated using the NORM.S.INV function in the excel sheet.

2.
The +0.18% & -0.18% are called the Margins of Error

3.
The 68%, 95%, etc. are called the Confidence Level
The Significance Level is calculated by subtracting the confidence level from 1.
So, if we choose to use the 95% confidence level, the Significance level = 1 - 95% = 0.05

4.
The ranges; 4.91% to 5.09%, and 4.81% to 5.18%; are called Confidence Interval

5.
p-value is the measurement of Statistical Significance of any given experiment's result.
It is also called the measurement of the Uncertainty of any given experiment's result.

6.
The 95% confidence level is the most preferred one for declaring the results of Ab tests.
The 99.7% is used where you are testing something which is extremely-sensitive-for-business.
Similarly, 0.05 is the most commonly used p-value to check whether the result is Statistically Significant or not.

7.
The existing red color button is called the Control.
The black color button, that we wanted to test, is called the Variant.

8.
In AB testing we start with 2 hypotheses:
Null Hypothesis (H0) - This says that the Control & Variant have no impact on the KPI (CTR, in our case)
Alternate Hypothesis (Ha) This says that the Control & Variant have different impacts on the KPI.

AB testing, hence, is used to check which hypothesis is correct.

If the p-value is less than the Significance level, i.e. we have got a high Confidence level, we say that our experiment has been successful i.e. Ha has come out to be true i.e. we reject the H0.

Similarly, if the p-value is more than or equal to the Significance level, i.e. we have got a low Confidence level, we say that our experiment has been a failure i.e. Ha has turned out to be false and we reject the same.

9.
A low standard deviation tells us that the data is closely clustered around the mean (or average) - and hence produces a skewed/pointed graph, while a high standard deviation indicates that the data is dispersed over a wider range of values - and hence produces a flattened/spread graph

10.
If the confidence intervals of your original page and variation b overlap, you need to keep testing even if your testing tool is saying that one is a statistically significant winner.


Further reading:
KhanAcademy.org/math/ap-statistics/tests-significance-ap/idea-significance-tests/v/p-values-and-significance-tests
VWO.com/blog/what-you-really-need-to-know-about-mathematics-of-ab-split-testing/
ConversionSciences.com/ab-testing-statistics/
YouTube.com/watch?v=cgxPcdPbujI
YouTube.com/watch?v=hlM7zdf7zwU
YouTube.com/watch?v=-MKT3yLDkqk