Mediaboss Marketing

AI, Artificial Intelligence concept,3d rendering,conceptual image.

This week IBM announced its latest Z-series mainframe and MLCommons released its latest benchmark series. The two announcements had something in common Machine Learning (ML) acceleration which is becoming pervasive everywhere from financial fraud detection in mainframes to detecting wake words in home appliances.

While these two announcements were not directly related, but they are part of a trend, showing how pervasive ML has become.

MLCommons Brings Standards to ML Benchmarking

ML benchmarking is important because we often hear about ML performance in terms of TOPS trillions of operations per second. Like MIPS (Millions of Instructions per Second or Meaningless Indication of Processor Speed depending on your perspective), TOPS is a theoretical number calculated from the architecture, not a measured rating based on running workloads. As such, TOPS can be a deceiving number because it does not include the impact of the software stack., Software is the most critical aspect of implementing ML and the efficiency varies widely, which Nvidia clearly demonstrated by improving the performance of its A100 platform by 50% in MLCommons benchmarks over the years.

The industry organization MLCommons was created by a consortium of companies to build a standardized set of benchmarks along with a standardized test methodology that allows different machine learning systems to be compared. The MLPerf benchmark suites from MLCommons include different benchmarks that cover many popular ML workloads and scenarios. The MLPerf benchmarks addresses everything from the tiny microcontrollers used in consumer and IoT devices, to mobile devices like smartphones and PCs, to edge servers, to data center-class server configuration. Supporters of MLCommons include Amazon, Arm, Baidu, Dell Technologies, Facebook, Google, Harvard, Intel, Lenovo, Microsoft, Nvidia, Stanford and the University of Toronto.

MLCommons releases benchmark results in batches and has different publishing schedules for inference and for training. The latest announcement was for version 2.0 of the MLPerf Inference suite for data center and edge servers, version 2.0 for MLPerf Mobile, and version 0.7 for MLPerf Tiny for IoT devices.

To date, the company that has had the most consistent set of submissions, producing results every iteration, in every benchmark test, and by multiple partners, has been Nvidia. Nvidia and its partners appear to have invested enormous resources in running and publishing every relevant MLCommons benchmark. No other vendor can match that claim. The recent batch of inference benchmark submissions include Nvidia Jetson Orin SoCs for edge servers and the Ampere-based A100 GPUs for data centers. Nvidias Hopper H100 data center GPU, which was announced at Spring 2022 GTC, arrived too late to be included in the latest MLCommons announcement, but we fully expect to see Nvidia H100 results in the next round.

Recently, Qualcomm and its partners have been posting more data center MLPerf benchmarks for the companys Cloud AI 100 platform and more mobile MLPerf benchmarks for Snapdragon processors. Qualcomms latest silicon has proved to be very power efficient in data center ML tests, which may give it an edge on power-constrained edge server applications.

Many of the submitters are system vendors using processors and accelerators from silicon vendors like AMD, Andes, Ampere, Intel, Nvidia, Qualcomm, and Samsung. But many of the AI startups have been absent. As one consulting company, Krai, put it: Potential submitters, especially ML hardware startups, are understandably wary of committing precious engineering resources to optimizing industry benchmarks instead of actual customer workloads. But then Krai countered their own objection with MLPerf is the Olympics of ML optimization and benchmarking. Still, many startups have not invested in producing MLCommons results for various reasons and that is disappointing. Theres also not enough FPGA vendors participating in this round.

The MLPerf Tiny benchmark is designed for very low power applications such as keyword spotting, visual wake words, image classification, and anomaly detection. In this case we see results from a mix of small companies like Andes, Plumeria, and Syntiant, as well as established companies like Alibaba, Renesas, Silicon Labs, and STMicroeletronics.

IBM z16 Mainframe

IBM Adds AI Acceleration Into Every Transaction

While IBM didnt participate in MLCommons benchmarks, the company takes ML seriously. With its latest Z-Series mainframe computer, the z16, IBM has added accelerators for ML inference and quantum-safe secure boot and cryptography. But mainframe systems have different customer requirements. With roughly 70% of banking transactions (on a value basis) running on IBM mainframes, the company is anticipating the needs of financial institutes for extreme reliable and transaction processing protection. In addition, by adding ML acceleration into its CPU, IBM can offer per-transaction ML intelligence to help detect fraudulent transactions.

In an article I wrote in 2018, I said: In fact, the future hybrid cloud compute model will likely include classic computing, AI processing, and quantum computing. When it comes to understanding all three of those technologies, few companies can match IBMs level of commitment and expertise. And the latest developments in IBMs quantum computing roadmap and the ML acceleration in the z16, show IBM is a leader in both.

Summary

Machine Learning is important from tiny devices up to mainframe computers. Accelerating this workload can be done on CPUs, GPUs, FPGAs, ASICs, and even MCUs and is now a part of all computing going forward. These are two examples of how ML is changing and improving over time.

Tirias Research tracks and consults for companies throughout the electronics ecosystem from semiconductors to systems and sensors to the cloud. Members of the Tirias Research team have consulted for IBM, Nvidia, Qualcomm, and other companies throughout the AI ecosystems.

Read the original:
IBM And MLCommons Show How Pervasive Machine Learning Has Become - Forbes

Unsupported sleeper detection

From the machine model development for detecting unsupported sleepers, the accuracy of each model is shown in Table 4.

From the table, it can be seen that each model performs well. The accuracy of each model is higher than 90% when the data processing is appropriate. CNN performs the best based on its accuracies. When CNN is applied with FFT and padding, the accuracies are the first and second highest compared to other models. For RNN and ResNet, the accuracies are higher than 90% when specific data processing is used. However, the accuracies become 80% approximately when another data processing technique is used. For FCN, data processing is not needed. The FCN model can achieve an accuracy of 95%. From the table, the models with the highest accuracy are CNN, RNN, FCN, and ResNet respectively. The complicated architecture of ResNet does not guarantee the highest accuracy. Moreover, the training time of ResNet (46s/epoch) is the longest followed by RNN (6s/epoch), FCN (2s/epoch), and CNN (1s/epoch) respectively. It can be concluded that the CNN model is the best model to detect supported sleepers in this study because it provides the highest accuracy or 100% while the training time is the lowest. At the same time, easy data processing likes padding is good enough to provide a good result. It is better than FFT in the CNN model which requires longer data processing. The accuracy of testing data of each model is shown in Fig.8.

Accuracies of testing data on unsupported sleeper detection.

The tuned hyperparameters of the CNN model with padding data are shown in Table 5.

Compared to the previous study, Sysyn et al.1 applied statistical methods and KNN which provided the best detection accuracy of 65%. The accuracy of the CNN model developed in this study is significantly higher. It can be assumed that the machine learning techniques used in this study are more powerful than the ones used in the previous study. Moreover, CNN is proven that it is suitable for pattern recognition.

For the unsupported sleeper severity classification, the performance of each model is shown in Table 6.

From the table, it can be seen that the CNN model still performs the best with an accuracy of 92.89% and provides good results with both data processing. However, the accuracies of RNN and ResNet significantly drop when unsuitable data processing is conducted. For example, the accuracy of the RNN model with padding drops to 33.89%. The best performance that RNN can achieve is 71.56% which is the lowest compared to other models. This is because of the limitation of RNN that vanishing gradient occurs when time-series data is too long. In this study, the number of data points for padding data is 1181 which can result in the issue. Therefore, RNN does not perform well. ResNet performs well with an accuracy of 92.42% close to CNN while the accuracy of FCN is fairly well. For the training time, CNN is the fastest model with the training time of 1s/epoch followed by FCN (2s/epoch), RNN (5s/epoch), and ResNet (32s/epoch) respectively. From these, it can be concluded that the CNN model is the best model for unsupported sleeper severity classification in this study. Moreover, it can be concluded that CNN and ResNet are suitable with padding data while RNN is suitable with FFT data. The accuracy of testing data of each model is shown in Fig.9.

Accuracies of testing data on unsupported sleeper severity classification.

The confusion matrix of the CNN model is shown in Table 7.

To clearly demonstrate the performance of each model, precision and recall are shown in Table 8.

From the table, the precisions and recalls of CNN and ResNet are fairly good with values higher than 80% while RNN is the worst. Some precisions of RNN are lower than 60% which cannot be used in realistic situations. CNN seems to be the better model than ResNet because all precisions are higher than 90%. Although some precisions of ResNet are higher than CNN, the precision of class 2 is about 80%. Therefore, the use of the CNN model is better.

For hyperparameter tuning, the tuned hyperparameters of CNN are shown in Table 9.

Originally posted here:
Prognostics of unsupported railway sleepers and their severity diagnostics using machine learning | Scientific Reports - Nature.com

Many higher-education institutions are now using data and analytics as an integral part of their processes. Whether the goal is to identify and better support pain points in the student journey, more efficiently allocate resources, or improve student and faculty experience, institutions are seeing the benefits of data-backed solutions.

Those at the forefront of this trend are focusing on harnessing analytics to increase program personalization and flexibility, as well as to improve retention by identifying students at risk of dropping out and reaching out proactively with tailored interventions. Indeed, data science and machine learning may unlock significant value for universities by ensuring resources are targeted toward the highest-impact opportunities to improve access for more students, as well as student engagement and satisfaction.

For example, Western Governors University in Utah is using predictive modeling to improve retention by identifying at-risk students and developing early-intervention programs. Initial efforts raised the graduation rate for the universitys four-year undergraduate program by five percentage points between 2018 and 2020.

Yet higher education is still in the early stages of data capability building. With universities facing many challenges (such as financial pressures, the demographic cliff, and an uptick in student mental-health issues) and a variety of opportunities (including reaching adult learners and scaling online learning), expanding use of advanced analytics and machine learning may prove beneficial.

Below, we share some of the most promising use cases for advanced analytics in higher education to show how universities are capitalizing on those opportunities to overcome current challenges, both enabling access for many more students and improving the student experience.

Data science and machine learning may unlock significant value for universities by ensuring resources are targeted toward the highest-impact opportunities to improve access for more students, as well as student engagement and satisfaction.

Advanced-analytics techniques may help institutions unlock significantly deeper insights into their student populations and identify more nuanced risks than they could achieve through descriptive and diagnostic analytics, which rely on linear, rule-based approaches (Exhibit 1).

Exhibit 1

Advanced analyticswhich uses the power of algorithms such as gradient boosting and random forestmay also help institutions address inadvertent biases in their existing methods of identifying at-risk students and proactively design tailored interventions to mitigate the majority of identified risks.

For instance, institutions using linear, rule-based approaches look at indicators such as low grades and poor attendance to identify students at risk of dropping out; institutions then reach out to these students and launch initiatives to better support them. While such initiatives may be of use, they often are implemented too late and only target a subset of the at-risk population. This approach could be a good makeshift solution for two problems facing student success leaders at universities. First, there are too many variables that could be analyzed to indicate risk of attrition (such as academic, financial, and mental health factors, and sense of belonging on campus). Second, while its easy to identify notable variance on any one or two variables, it is challenging to identify nominal variance on multiple variables. Linear, rule-based approaches therefore may fail to identify students who, for instance, may have decent grades and above-average attendance but who have been struggling to submit their assignments on time or have consistently had difficulty paying their bills (Exhibit 2).

Exhibit 2

A machine-learning model could address both of the challenges described above. Such a model looks at ten years of data to identify factors that could help a university make an early determination of a students risk of attrition. For example, did the student change payment methods on the university portal? How close to the due date does the student submit assignments? Once the institution has identified students at risk, it can proactively deploy interventions to retain them.

Though many institutions recognize the promise of analytics for personalizing communications with students, increasing retention rates, and improving student experience and engagement, institutions could be using these approaches for the full range of use cases across the student journeyfor prospective, current, and former students alike.

For instance, advanced analytics can help institutions identify which high schools, zip codes, and counties they should focus on to reach prospective students who are most likely to be great fits for the institution. Machine learning could also help identify interventions and support that should be made available to different archetypes of enrolled students to help measure and increase student satisfaction. These use cases could then be extended to providing students support with developing their skills beyond graduation, enabling institutions to provide continual learning opportunities and to better engage alumni. As an institution expands its application and coverage of advanced-analytics tools across the student life cycle, the model gets better at identifying patterns, and the institution can take increasingly granular interventions and actions.

Institutions will likely want to adopt a multistep model to harness machine learning to better serve students. For example, for efforts aimed at improving student completion and graduation rates, the following five-step technique could generate immense value:

Institutions could deploy this model at a regular cadence to identify students who would most benefit from additional support.

Institutions could also create similar models to address other strategic goals or challenges, including lead generation and enrollment. For example, institutions could, as a first step, analyze 100 or more attributes from years of historical data to understand the characteristics of applicants who are most likely to enroll.

Institutions will likely want to adopt a multistep model to harness machine learning to better serve students.

The experiences of two higher education institutions that leaned on advanced analytics to improve enrollment and retention reveal the impact such efforts can have.

One private nonprofit university had recently enrolled its largest freshman class in history and was looking to increase its enrollment again. The institution wanted to both reach more prospective first-year undergraduate students who would be a great fit for the institution and improve conversion in the enrollment journey in a way that was manageable for the enrollment team without significantly increasing investment and resources. The university took three important actions:

For this institution, advanced-analytics modeling had immediate implications and impact. The initiative also suggested future opportunities for the university to serve more freshmen with greater marketing efficiency. When initially tested against leads for the subsequent fall (prior to the application deadline), the model accurately predicted 85 percent of candidates who submitted an application, and it predicted the 35 percent of applicants at that point in the cycle who were most likely to enroll, assuming no changes to admissions criteria (Exhibit 3). The enrollment management team is now able to better prioritize its resources and time on high-potential leads and applicants to yield a sizable class. These new capabilities will give the institution the flexibility to make strategic choices; rather than focus primarily on the size of the incoming class, it may ensure the desired class size while prioritizing other objectives, such as class mix, financial-aid allocation, or budget savings.

Exhibit 3

Similar to many higher-education institutions during the pandemic, one online university was facing a significant downward trend in student retention. The university explored multiple options and deployed initiatives spearheaded by both academic and administrative departments, including focus groups and nudge campaigns, but the results fell short of expectations.

The institution wanted to set a high bar for student success and achieve marked and sustainable improvements to retention. It turned to an advanced-analytics approach to pursue its bold aspirations.

To build a machine-learning model that would allow the university to identify students at risk of attrition early, it first analyzed ten years of historical data to understand key characteristics that differentiate students who were most likely to continueand thus graduatecompared with those who unenrolled. After validating that the initial model was multiple times more effective at predicting retention than the baseline, the institution refined the model and applied it to the current student population. This attrition model yielded five at-risk student archetypes, three of which were counterintuitive to conventional wisdom about what typical at-risk student profiles look like (Exhibit 4).

Exhibit 4

Together, these three counterintuitive archetypes of at-risk studentswhich would have been omitted using a linear analytics approachaccount for about 70 percent of the students most likely to discontinue enrollment. The largest group of at-risk individuals (accounting for about 40 percent of the at-risk students identified) were distinctive academic achievers with an excellent overall track record. This means the model identified at least twice as many students at risk of attrition than models based on linear rules. The model outputs have allowed the university to identify students at risk of attrition more effectively and strategically invest in short- and medium-term initiatives most likely to drive retention improvement.

With the model and data on at-risk student profiles in hand, the online university launched a set of targeted interventions focused on providing tailored support to students in each archetype to increase retention. Actions included scheduling more touchpoints with academic and career advisers, expanding faculty mentorship, and creating alternative pathways for students to satisfy their knowledge gaps.

Advanced analytics is a powerful tool that may help higher-education institutions overcome the challenges facing them today, spur growth, and better support students. However, machine learning is complex, with considerable associated risks. While the risks vary based on the institution and the data included in the model, higher-education institutions may wish to take the following steps when using these tools:

While many higher-education institutions have started down the path to harnessing data and analytics, there is still a long way to go to realizing the full potential of these capabilities in terms of the student experience. The influx of students and institutions that have been engaged in online learning and using technology tools over the past two years means there is significantly more data to work with than ever before; higher-education institutions may want to start using it to serve students better in the years to come.

Here is the original post:
Machine learning in higher education - McKinsey

LOS ANGELES, April 12, 2022 (GLOBE NEWSWIRE) -- Mission, a managed cloud services provider and Amazon Web Services (AWS) Premier Services Partner, today announced the company has won a 2021 Top Projects Award from TechTargets SearchITChannel. The annual award honors three IT services partners and their customers for exceptional technological initiatives that demonstrate compelling innovation, creative partnering, and business-wide benefits.

JibJab sought support from an AWS partner to achieve its goals around image quality and customer experience as it prepared to launch its user-designed Starring You Books. For the iconic digital entertainment studio known for enabling users to send personalized e-cards, the books would mark the companys first expansion into a physical product line. During the projects initial planning process, JibJab realized the opportunity to utilize a machine learning computer vision algorithm to detect faces within user-uploaded photos. The algorithm would need to automatically crop faces and hair from photos and perform post-processing to prepare print-quality images. Without the in-house ML expertise to build this algorithm and wanting to avoid the cost-prohibitive licensing fees of using an existing ML algorithm JibJab partnered with Mission to develop and complete the project.

Mission leveraged its AWS machine learning expertise to build and train the algorithm, implementing a process that included data labeling and augmentation with a training set of 17,000 images. Experts from Missions Data, Analytics & Machine Learning practice created JibJabs solution using several solutions, including Amazon SageMaker, Amazon Rekognition, and Facebooks Detectron2. This work has resulted in a seamless self-service experience for JibJab customers, who can upload their photos and have final, book-ready images prepared by the ML algorithm in just five seconds. Customers then simply place the final images within their personalized Starring You Books products using a GUI, and approve their work for printing.

Quotes

We talked to a few external companies and Mission was our clear preference, said Matt Cielecki, VP of Engineering at JibJab. It became evident from day one that Mission wasnt just going to throw something over the fence for us to use; the team was going to ensure that we understood the rationale behind the processes and technologies put into action.

Missions work with JibJab showcases the tremendous potential AWS and ML can enable for developing innovative new products and unprecedented customer experiences, said Ryan Ries, Practice Lead, Data Science & Engineering at Mission.We jumped at the opportunity to work with JibJab on this project and are proud of the success of the project and to have the work recognized with TechTarget SearchITChannels 2021 Top Projects Award.

About Mission Cloud Services

Mission accelerates enterprise cloud transformation by delivering a differentiated suite of agile cloud services and consulting. As an AWS Premier Services Partner, Missions always-on services enable businesses to scale and outpace competitors by leveraging the most transformative technology platform and enterprise software ecosystem in history.

ContactKyle Petersonkyle@clementpeterson.com

A photo accompanying this announcement is available at https://www.globenewswire.com/NewsRoom/AttachmentNg/d7325672-6f04-42ed-8959-9d365045ea72

Read more from the original source:
Mission Cloud Services Wins TechTarget Award for its Innovative AWS Machine Learning Work with JibJab - GlobeNewswire

What am I doing here? The story so far

As you might know if you have read our blog for more than a year, a few years ago, I bought a flat in Paris. If you don't know, the real estate market in Paris is expensive but despite that, it is so tight that a good flat at a correct price can be for sale for less than a day.

Obviously, you have to take a decision quite fast, and considering the prices, you have to trust your decision. Of course, to trust your decision, you have to take your time, study the market, make some visits etc This process can be quite long (in my case it took a year between the time I decided that I wanted to buy a flat and the time I actually commited to buying my current flat), and even spending a lot of time will never allow you to have a perfect understanding of the market. What if there was a way to do that very quickly and with a better accuracy than with the standard process?

As you might also know if you are one of our regular readers, I tried to solve this problem with Machine Learning, using an end-to-end software called Dataiku. In a first blog post, we learned how to make a basic use of Dataiku, and discovered that just knowing how to click on a few buttons wasn't quite enough: you had to bring some sense in your data and in the training algorithm, or you would find absurd results.

In a second entry, we studied a bit more the data, tweaked a few parameters and values in Dataiku's algorithms and trained a new model. This yielded a much better result, and this new model was - if not accurate - at least relevant: the same flat had a higher predicted place when it was bigger or supposedly in a better neighbourhood. However, it was far from perfect and really lacked accuracy for several reasons, some of them out of our control.

However, all of this was done on one instance of Dataiku - a licensed software - on a single VM. There are multiple reasons that could push me to do things differently:

What we did very intuitively (and somewhat naively) with Dataiku was actually a quite complex pipeline that is often called ELT, for Extract, Load and Transform.

And obviously, after this ELT process, we added a step to train a model on the transformed data.

So what are we going to do to redo all of that without Dataiku's help?

When ELT becomes ELTT

Now that we know what we are going to do, let us proceed!

Before beginning, we have to properly set up our environment to be able to launch the different tools and products. Throughout this tutorial, we will show you how to do everything with CLIs. However, all these manipulations can also be done on OVHcloud's manager (GUI), in which case you won't have to configure these tools.

For all the manipulations described in the next phase of this article, we will use a Virtual Machine deployed in OVHcloud's Public Cloud that will serve as the extraction agent to download the raw data from the web and push it to S3 as well as a CLI machine to launch data processing and notebook jobs. It is a d2-4 flavor with 4GB of RAM, 2 vCores and 50 GB of local storage running Debian 10, deployed in Graveline's datacenter. During this tutorial, I run a few UNIX commands but you should easily be able to adapt them to whatever OS you use if needed. All the CLI tools specific to OVHcloud's products are available on multiple OSs.

You will also need an OVHcloud NIC (user account) as well as a Public Cloud Project created for this account with a quota high enough to deploy a GPU (if that is not the case, you will be able to deploy a notebook on CPU rather than GPU, the training phase will juste take more time). To create a Public Cloud project, you can follow these steps.

Here is a list of the CLI tools and other that we will use during this tutorial and why:

Additionally you will find commented code samples for the processing and training steps in this Github repository.

In this tutorial, we will use several object storage buckets. Since we will use the S3 API, we will call them S3 bucket, but as mentioned above, if you use OVHcloud standard Public Cloud Storage, you could also use the Swift API. However, you are restricted to only the S3 API if you use our new high-performance object storage offer, currently in Beta.

For this tutorial, we are going to create and use the following S3 buckets:

To create these buckets, use the following commands after having configured your aws CLI as explained above:

Now that you have your environment set up and your S3 buckets ready, we can begin the tutorial!

First, let us download the data files directly on Etalab's website and unzip them:

You should now have the following files in your directory, each one corresponding to the French real estate transaction of a specific year:

Now, use the S3 CLI to push these files in the relevant S3 bucket:

You should now have those 5 files in your S3 bucket:

What we just did with a small VM was ingesting data into a S3 bucket. In real-life usecases with more data, we would probably use dedicated tools to ingest the data. However, in our example with just a few GB of data coming from a public website, this does the trick.

Now that you have your raw data in place to be processed, you just have to upload the code necessary to run your data processing job. Our data processing product allows you to run Spark code written either in Java, Scala or Python. In our case, we used Pyspark on Python. Your code should consist in 3 files:

Once you have your code files, go to the folder containing them and push them on the appropriate S3 bucket:

Your bucket should now look like that:

You are now ready to launch your data processing job. The following command will allow you to launch this job on 10 executors, each with 4 vCores and 15 GB of RAM.

Note that the data processing product uses the Swift API to retrieve the code files. This is totally transparent to the user, and the fact that we used the S3 CLI to create the bucket has absolutely no impact. When the job is over, you should see the following in your transactions-ecoex-clean bucket:

Before going further, let us look at the size of the data before and after cleaning:

As you can see, with ~2.5 GB of raw data, we extracted only ~10 MB of actually useful data (only 0,4%)!! What is noteworthy here is that that you can easily imagine usecases where you need a large-scale infrastructure to ingest and process the raw data but where one or a few VMs are enough to work on the clean data. Obviously, this is more often the case when working with text/structured data than with raw sound/image/videos.

Before we start training a model, take a look at these two screenshots from OVHcloud's data processing UI to erase any doubt you have about the power of distributed computing:

In the first picture, you see the time taken for this job when launching only 1 executor- 8:35 minutes. This duration is reduced to only 2:56 minutes when launching the same job (same code etc) on 4 executors: almost 3 times faster. And since you pay-as-you go, this will only cost you ~33% more in that case for the same operation done 3 times faster- without any modification to your code, only one argument in the CLI call. Let us now use this data to train a model.

To train the model, you are going to use OVHcloud AI notebook to deploy a notebook! With the following command, you will:

In our case, we launch a notebook with only 1 GPU because the code samples we provide would not leverage several GPUs for a single job. I could adapt my code to parallelize the training phase on multiple GPUs, in which case I could launch a job with up to 4 parallel GPUs.Once this is done, just get the URL of your notebook with the following command and connect to it with your browser:

Once you're done, just get the URL of your notebook with the following command and connect to it with your browser:

You can now import the real-estate-training.ipynb file to the notebook with just a few clicks. If you don't want to import it from the computer you use to access the notebook (for example if like me you use a VM to work and have cloned the git repo on this VM and not on your computer), you can push the .ipynb file to your transactions-ecoex-clean or transactions-ecoex-model bucket and re-synchronize the bucket to your notebook while it runs by using the ovhai notebook pull-data command. You will then find the notebook file in the corresponding directory.

Once you have imported the notebook file to your notebook instance, just open it and follow the directives. If you are interested in the result but don't want to do it yourself, let's sum up what the notebook does:

Use the models built in this tutorial at your own risk

So, what can we conclude from all of this? First, even if the second model is obviously better than the first, it is still very noisy: while not far from correct on average, there is still a huge variance. Where does this variance come from?

Well, it is not easy to say. To paraphrase the finishing part of my last article:

In this article, I tried to give you a glimpse at the tools that Data Scientists commonly use to manipulate data and train models at scale, in the Cloud or on their own infrastructure:

Hopefuly, you now have a better understanding on how Machine Learning algorithms work, what their limitations are, and how Data Scientists work on data to create models.

As explained earlier, all the code used to obtain these results can be found here. Please don't hesitate to replicate what I did or adapt it to other usecases!

Solutions ArchitectatOVHCloud|+ posts

Here is the original post:
OVH Groupe : A journey into the wondrous land of Machine Learning, or Cleaning data is funnier than cleaning my flat! (Part 3) - Marketscreener.com