The 5 Most Critical Insights You’ll Gain in Your First 5 Years as a Data Scientist

Your first five years as a data scientist are going to feel a lot like drinking from a firehose.

New information is going to be coming from left and right, you’re going to have to re-learn some of the things that you learned at the very beginning of your data science journey, and you’re going to have to figure out your place within the company.

By the end of your first five years, you may feel as though you’ve been hit by a bus of information and learning experiences.

However, when you look back on those first five years, you’ll probably notice something pretty cool: you’ve gained several valuable insights that you can use to leverage your career in the next five years.

Read more at Towards Data Science

Openbridge Review

This is the next post in our pursuit of an Amazon Automation tool.

As mentioned in our prior post, during a build or buy assessment, there were two platforms reviewed for the "buy" side of the analysis; Openbridge and Saras Analytics Daton.  See our review of Saras Analytics Daton.

During the evaluation, Openbridge released a new user interface for their SaaS data app. This caused us to have to redo our review given the new app was a big change from the old version.  To put a finer point on the update, the new user interface was a significant improvement over the last version, for beginners and pros alike. While the old interface was clunky, the new one is simple, clean, and modern, with a "darker" theme which was a pleasant surprise. Setup was simple, guided, and without incident. 

Unlike Saras, when asked Openbridge confirmed they only use official, approved Amazon APIs. No bots. This makes sense since they are a verified Amazon Selling Partner and Amazon Advertising developer. They even shared a "bot free promise" that highlights the integration philosophy they follow.

There is some data that Amazon does not make available via API, which means Openbridge will not offer it. Saras on the other hand uses bots, which allows them to screen scrape data from your account for these reports. Our guess is Saras made the decision to use bots to deliver access to data not available from Amazon's API given most customers would be unaware of the risks. Not very transparent.

Openbridge has been certified by Amazon for PII data, which reinforces the commitment to following Amazon's best practices. Saras Analytics Daton use of bots indicates a willingness to provide data not yet officially available via API. The desire to go beyond the API is a violation of the spirit and letter of the terms of service. It is not clear why Saras would take such commercial risks given Saras is not certified by Amazon and likely will not be if they continue to bypass Amazon terms with bots.

Overall, Openbridge produced consistent and reliable outputs. We used Snowflake, BigQuery, and Amazon Athena; all worked without incident. This is likely the result of each connector being uniquely pre-modeled to the best practices of the Amazon API, which means there is little configuration expected of the user. For example, Openbridge does not allow a user to configure an invalid schedule like Daton does. They simplified most of the process to simply selecting a source and they pointing it to a private destination. 

As a team, they seem deeply rooted in the Amazon ecosystem, which was good to know. Amazon can be complex, so having a platform and data engineering expertise at hand is a plus. They do not offer any reporting templates or dashboards for tools like Tableau or Power BI. More on that in a bit.

Different from other platforms we have investigated was the new Openbridge pricing. The pricing is completely usage-based.  Usage is based on how many accounts you want to collect data from. This makes predicting costs over the course of a year simple. There are no row costs, just total your accounts and connectors to know the costs. Also, they allow you to turn something off, which means you no longer incur usage charges. The finance team loves the ability to forecast a price with certainty for the fiscal year, so this is a plus.

Both Saras Analytics Daton and  Openbridge offer solutions for Sellers and Vendors. Openbridge does not offer any BI or reports, just the "ETL" of data to a destination. They detail why in this support doc Analytics-Ready: Freedom to Connect Your Data Tools. However, Openbridge brings the proper balance of function of performance, cost, security, and reliability. Their commitment to adhering to the letter of their Amazon partner agreements makes them the right decision for Amazon automation efforts.

Openbridge vs Saras Analytics Daton

ETL Tools Comparison - Daton

Daton Review - Saras Analytics

If you are a Seller or Vendor, data is crucial. Being a data-driven merchant means automating your data workflows. Manual data processing is messy, confusing, and time-consuming. There are many tools on the market that allow you to connect to Amazon. We discovered that it is important to ask how these tools connect to your Seller or Vendor account is an important question to ask.

During a build or buy assessment, there were two platforms were reviewed for the "buy" side of the analysis; Openbridge and Saras Analytics Daton.  See our review of Openbridge.

The sign-up process for Daton was quick, only taking a few minutes. The one caveat is the "bot" setup process, which can take a few days. More on that later.

In terms of getting data flowing, the process was overall easy to understand. There were some issues post setup, which are detailed later. Generally, this had to do with the UI allowing what are invalid configurations according to Amazon documentation.

One of the major concerns with Saras Analytics Daton was the use of data scraping, or as they call it "robotic process automation". Using bots as a data automation process is fragile, and a major security red flag. Cloudflare details this in a post that "attackers can use web scraping tools to access data much more rapidly than intended. This can result in data being used for unauthorized purposes." As a result, data scraping can open the door to all sorts of issues, including a comprised Amazon account. See Data Scraping – Considering the Privacy Issues and Data Scraping: Associated Security and Privacy Risks for more context. Also, the International Association of Privacy Professionals (IAPP) indicated bots can present GDPR compliance issues. Given all the headaches, they normally should be avoided, especially from a commercial provider like Saras Analytics.

Using a test account, the surface area of any risks related to the use of bots was minimized ( a prod account should never be linked to anyone using bots as it is a violation of Amazon terms ). Even using a test account, the idea that some bot processes were running was uncomfortable. However, using the test account allowed us to get an overall sense of the Saras Analytics Daton platform, while not incurring any undue worry. 

As we stated earlier, the config process was generally consistent and easy to understand, but the process was not reliable. The UI allowed us to configure integration to Amazon that Amazon stated was not allowed or invalid. For example, the UI allowed configuring a sync every 60 minutes for data that is only available daily. As a matter of fact, the misconfiguration led to our Amazon account becoming unstable, as the requests from Daton were blocking our ERP from working. Not good. Also, we observed inconsistent and unreliable syncs in other cases as well.

The pricing for their service uses row counts, which can be very difficult to estimate.  Attempting to project a budget over 12 months was going to require a cost-plus analysis.  What is cost-plus pricing? It requires you to determine the base cost of the product and then add a percentage on top of that price use costs. It is not uncommon for row-based billing schemes to be 2x to 10x your base product cost.

Saras Analytics, not Daton, does offer consulting and analytics dashboard templates. If you want pre-designed reporting templates, this may be of interest. The idea of dashboard templates may be appealing to teams looking to have something pre-designed. Openbridge does not offer data visualization or reporting templates. They focus on data engineering, referring anyone that needs Tableau, Data Studio, or Power BI dashboard and reporting to their partner network.

Both Saras Analytics Daton and Openbridge offer solutions for Sellers and Vendors. However, choosing the right one is a function of performance, cost, security, and reliability.

While Saras Analytics Daton offered a few report data feeds that Openbridge did not the risks and costs associated with using the service to get those reports were not worth it. The new Openbridge platform, pricing, and commitment to adhering to the letter of their Amazon partner agreements instilled confidence that selecting them was the right decision.

References:

Openbridge vs Saras Analytics Daton

Openbridge vs Saras Analytics

ETL Tools Comparison - Daton