Data Science-Product Analysis and Business Metrics Interview Questions

1. In 2011 Facebook launched messenger as a standalone app for mobile devices (it used to only be part of the Facebook App). How would you track the performance of this new application? What metrics would you use?

Answer: Well, there are lots of different possible answers. This is an example answer, instead of 100% correct solution. Let’s get started by thinking about some metrics.

• So, one common metrics is DAILY ACTIVE USERS and MONTHLY ACTIVE USERS: There are Daily and monthly active users. In this case we can use the before and after DAILY ACTIVE USERS and MONTHLY ACTIVE USERS to compare if users have moved on to the new app.
• Another metrics is total time spent: This allows you to compare engagement before and after the launch of the standalone app. So time spent on app, DAILY ACTIVE USERS, and MONTHLY ACTIVE USERS are common metrics you can use for any sort of application or website analysis. Let’s now focus on metrics that may be more specific to the messenger App itself.
• So some metrics we want to consider are the actual number of messages being sent before and after the Check response time to messages received. Keep in mind increased metrics can also sometimes be deceiving. So some question are – is it possible more time is being spent on the app because it is harder to use? Could this possibly affect advertising revenue changes? How is the previous Facebook App affected by the change?

Want to know More about Data Science? Click here

2. Consider a scenario, Google has released a new version of their search algorithm, for which they used A/B testing. During the testing process, engineers realized that the new algorithm was not implemented correctly and returned less relevant results. Two things happened during testing:

• People in the treatment group performed more queries than the control group.
• Advertising revenue was higher in the treatment group as well.
• So, the first question for this situation is:

What may be the cause of people in the treatment group performing more searches than the control group? There are different possible answers here.

Answer: We know the new algorithm produced less relevant search results. This means that users may have to make additional searches in order to clarify what they are searching for using the new algorithm.

In order to test this hypothesis will be could do a study how close searches are to each other. So, if we notice that additional searchers are being done very soon after a previous search we could classify them as clarifying searches. More queries do not mean that the new algorithm is better or worse it just means that it’s different. We have to do is look at this analytically and the side is it better or worse given what we do know and most likely since we do know that it’s giving less relevant results.

3. Consider the same situation, Google has released a new version of their search algorithm, for which they used A/B testing. During the testing process, engineers realized that the new algorithm was not implemented correctly and returned less relevant results. Two things happened during testing:

• People in the treatment group performed more queries than the control group.
• Advertising revenue was higher in the treatment group as well.

So, the second question is, What you think caused the new algorithm to generate more advertising revenue, even though the results were less relevant?

Answer: So, there are definitely two main things to cover here.

We know that more searches are being conducted, since advertisements are served along with every new search, there are more opportunities for users to click on the advertisement. So this in itself is not directly related to the fact that there are less relevant results. It’s just the fact that if more people are doing more searches that means more advertisements are being you’re probably going to produce more advertising revenue. Another possibility is that the search algorithm is different than the algorithm used display ads. In this case, the ads themselves may be more relevant than the search results, causing more ad clicks.

4. Consider again the same situation, Google has released a new version of their search algorithm, for which they used A/B testing. During the testing process, engineers realized that the new algorithm was not implemented correctly and returned less relevant results. Two things happened during testing:

• People in the treatment group performed more queries than the control group.
• Advertising revenue was higher in the treatment group as well. So the question is since the less relevant algorithm resulted in higher advertising revenue, should it be implemented anyways?

Answer: This is a bit of an opinion question, but we should probably not implement this new algorithm. The effects described are probably only short-term effects due to the problems with the algorithm.

We should not sacrifice the long term potential of the site for a temporary increase in revenue and searches. Google is probably best positioned to win in the long term when it has the most relevant search algorithm.

5. A car company produces all the cars for a country, we’ll call them Car X and Car Y. 50% of the population drives Car X, the other 50% drives Car Y. Two potential technologies have just been discovered that help reduce gasoline usage! There are two technologies:

• Technology A increases the MPG or mpg of car X from 50 MPG to 75 MPG.
• Technology B increases the mpg of car Y from 10 mpg to 11 miles per gallon.

So the question is which technologies should be implemented to save the most gasoline for the entire country?

Answer: So, this is actually a really interesting problem. Let us assume that the average commute distance for a car is D.

Then we define total gas used (G) as G= D/MPG. Let’s examine what each policy would do.

Total gas used change with Policy A:

(D/50)- (D/75) = D /150

Total gas used change with Policy B:

(D/10)-(D/11) = D/110

Since, we know that for any average commute distance D/150 will always be smaller than D/110, then Policy B results in the biggest gas change or savings. So Policy B is the correct choice.

6. What are the Essentials Business Skills for Data Science Success?

Answer:

• Learn best practices for using data analytics to make a business and to achieve its objectives.
• Learn to recognize the most critical business metrics and distinguish those from mere data.
• Clearly understand the roles played by Business Analysts, Business Data Analysts, and Data Scientists know exactly the skills required to be hired and to succeed in each of these.

7. What about Business Metrics?

Answer: There are three metrics: Revenue, Profitability and Risk Metrics.

Revenue metric is related to sales and marketing, Profitability is related to efficiency, operations, Risk is related to sustainability given present cash-flow conditions.

There are traditional Vs Dynamic Metrics. Any change is not easy to see in traditional metrics, like Quarterly revenue. It is easy to measure in dynamic metric like website visits getting converted to clicks and eventually purchase

8. What are Product Metrics?

Answer: Like any metric, they are quantitative measurements that help business workers to gain insights into the efficacy of their methods and the evolution of their project. In that specific case, they help in assessing your product performance by checking if it meets the original business goals and if the product strategy is working.

Without product KPIs, evaluating the performance of your product might very well end up in a guessing game where reality is skewed.

9. What is traditional Business Metric?

Answer: Traditional Metrics : Personal Sales.

Some key metrics are sales leads, qualified sales leads, time taken in getting to the right person etc., This is an example of not data driven metric

10. What is Dynamic Metrics ?

Answer: Computer Sales?—?Data Driven?—?This section covers Revenue Metrics. Dynamic metrics examples are click rates, most sought after items, people who viewed this items also viewed these etc.

11. What is Profitability/Efficiency Metrics?

Answer: Average number of days in inventory-called “days inventory”- is one of the Profitability Metric.

12. What is a Risk Metric?

Answer: Time to product recalls is a good risk metric.

13. What is the intent of project metrics?

a. Minimization of development schedule

b. For strategic purposes

c. Assessing project quality on ongoing basis

d. Both a and c

Answer: (d)

14. Which of the following is an indirect measure of product?

a. Complexity

b. Reliability

c. Quality

d. All of the Mentioned

Answer: (d) All of the Mentioned

15. A graphical technique for finding if changes and variation in metrics data are meaningful is known as

a. DRE (Defect Removal Efficiency)

b. Function points analysis

c. Control Chart

d. All of the mentioned

Answer: (c) Control Chart

16. Usability in metric analysis is defined as the degree to which the software:

a. stated needs

b. is easy to use

c. makes optimal use of system resources

d. none of the mentioned

Answer: b

17. In new product development process, after analysis of business next step to be taken is

a. test marketing

b.One channel marketing

c. penetration marketing

d. individual marketing

 Answer: a. test marketing

Learn Data Science from Industry Experts

18. When new developed product concept is tested, next immediate step is to

a. develop market strategy

b. develop a testing technique

c. develop intermediaries

d. develop logistic network

Answer: a. develop market strategy

19. Major sources of ideas for product development comes from

a. internal sources

b. external sources

c. product lines extension

d. both a and b

Answer: both a and b

20. An idea for a possible product that company will offer is classified as

a. product idea

b. product image

c. customer management

d. none of the above

Answer: a. product idea