Labeling and Data Augmentation with SageMaker

0:00Welcome back, beautiful learners to another skill. In 1769, Wolfgang von Kempl en wanted desperately to impress Queen Maria Theresa, and he vowed to do just that. Six months later, Kemplen proudly unveiled a contraption he dubbed the Turk. It was a machine capable of playing other humans in chess and defeating them pretty soundly. Well, the Queen was indeed impressed, as were many others who dared to challenge this mysterious humanoid device to a match.

0:30Throughout the late 1700s, in the first half of the 1800s, this mechanical Turk toured around the globe, defeating notable figures like Benjamin Franklin and Napoleon Bonap arte. The device was sold several times to new owners looking to bring this modern marvel to audiences around the world for a fee, of course. Some people regarded it as a technological breakthrough while others insisted that there had to be some supernatural powers at work. There were skeptics though, Edgar Allen Poe and Charles Babbage publicly denounced the device as a hoax, and they were right. It wasn't until 1857 that one of the sons of a former owner of the device came clean and admitted the truth.

1:14The mechanical Turk was in fact powered by humans. The owner would hire chess masters to hide inside the cabinet and play chess with human opponents using a series of pulleys and levers. Well, that was way back in the 1800s, surely the same stunt wouldn't fly today. Oh yes, oh yes it does. All the tech giants have at one point or another made use of click workers to either overtly or covertly supplement their respective AI capacities. Amazon does this through a service interestingly called Amazon Mechanical Turk. Back in 2015, Facebook did this covertly when they tried to launch a personal assistant called M touting cutting edge AI, but humans were really doing the work. And then there is the Nate shopping app in which its founder claimed that it had industry leading AI capabilities. And that helped him secure $40 million of investor money. Only problem was the AI capabilities were in fact call center workers. And as of this recording that has landed its founder in a very sticky situation. For all the marvelous technology that we have out there, there are still things that require humans. And in this skill, we're going to

2:33dig into a little of both with one of our data sets, a little bit of human and a little bit of machine. Let's get started.

0:00Okay, in our last skill, we talked about SageMaker, and I mentioned that we're kind of caught between two minds right now. We have the SageMaker next gen, and then we have SageMaker, which is kind of the traditional SageMaker has been around for a while, and AWS has named that SageMaker AI. And it's kind of unfortunate, but we find ourselves in this in-between world, and we have to know a little bit about both of these tools. Because over time, eventually, everything is going to gravitate more towards the SageMaker next gen.

0:34But in the meantime, these tools over here are still very popular, and they're used quite a bit. So there is going to be a huge transition over the course of probably a few years or so before everything is finally over here. Despite what Amazon wants you to do is drop everything over in the old world and lean into the new world, well, we are going to talk about one of those tools that, as far as I can tell, is not yet available in the new world. So if we look at SageMaker AI, we looked at SageMaker Studio, we looked at Can vas.

1:07We also have some tools here inside Canvas, where we can access it via Canvas, and that tool is called Data Wrangler. Now what is Data Wrangler? Well, it is yet another ETL tool that allows us to do some different stuff. It's a little bit more simplistic than glue or data brew, and it's more geared towards getting data in shape for true machine learning processes. So we're going to play around a little bit with this.

1:37We're not going to go too deep at all, because in all likelihood, if you're doing major machine learning workloads, you're probably not going to use Data Wrangler because it's kind of like the Fisher Price edition of being able to manipulate and augment your data . But it's pretty decent for what it is. And the real thing is that, yes, it is on the exam blueprint. So I have a responsibility to show you how it works and be sure that you know how to get hands on with this thing. Now something else we are going to use here is within the umbrella of SageMaker Ground Truth, and we are going to do some labeling.

2:17Now what is labeling? Well, I'm glad you asked. Let's get to it. So let's say that we are going to create an image classification model. Now there are several different ways to create image classification models. And one of the ways to do that is to create something called a convolutional neural network or a CNN. And so let's say I sketched this little drawing here, I made a little house, and we fed that into a computer, and guess what, that computer is going to digitize my drawing into all these little pixels and all these little blocks.

2:53So the next step we can take with that is we can say, hey, let's assign a weight or some sort of numeric value to whether this block is white or black. And so in this case over here, we have zero as white and we have one as black. This is a very simple color palette here. We're just using black and white and we can use these weights right here. Well, they're not really weights, they're just values. So now we have a numeric representation of what this little sketch looks like.

3:24And so what we can do is we can use a machine learning process and we can train a convolutional neural network to begin to understand these patterns and look for these patterns in other new drawings or new images that we may feed in. That process is called training. So we are giving the CNN model a bunch of data to look at and we're saying, hey , find more or get better at finding more of what this thing is.

3:55Part of finding what this thing is is we have to give this thing a name and that's called a label. So we would say, hey, this is a house. So what we'd want to do is have that model learn that this particular pattern of ones and zeros. You can kind of forget about the black and white here. Computers don't see that. They just see ones and zeros here. This particular pattern of ones and zeros equals a house. And so therefore in the future, if we feed this thing into our model, our model is going to say, hey, that sure looks really close to what you told me was a house before.

4:32So I'm going to say, hey, this is a house. And I think maybe I have 98% confidence that what you're giving me here, I've seen something like that before and I'm going to call it a house. That is how image classification works. It's not smoking mirrors. It's just all mathematics behind the scene. So if we were just saying, hey, try to detect a house in this image is this image a house, we could train our model on a bunch of different houses here and we can say, yes, it is a house or if it doesn't match, if it has no similarities to something that we've fed into our model before, the model is going to say, no, I don't think that that is a house at all.

5:15So this is something called binary classification. And it's called binary classification because there's only two choices here, you're either a house or you're not a house. And we also have something else. And that is a multiple classification. Most image classification models nowadays are able to do multiple class ifications. So in other words, you'll send in an image and maybe it's a house, maybe it's a cat, maybe it's a goat and that model will be able to say with a degree of certainty , whether it thinks it's a house, a cat, a goat, a car or whatever else that particular model was trained on.

5:52And then as multiple classifications, in some cases, some models can do multi label classification. And let's say that this is a house, but maybe it's a white house. And it's going to say, hey, that is a color white. I also know that is a house. Maybe it's a brick white house or something like that. So they can get pretty specific. So let's say we have our picture of a goat here and we train our model on this little guy right here. And then for goat, it has a mental image, it has a mathematical image, not a mental image.

6:25We have a mental image. Our model has a mathematical image, a bunch of ones and zeros as to what represents a picture of what we call a goat. The model doesn't know what it's looking at. It doesn't know that it's looking at an animal unless we have trained it and said, hey, not only is this a goat, this is also an animal. So it's going to start building those relationships. So what happens if we pass in a different picture? This is a different goat. It's still a goat, but it's from a different perspective.

6:57The goat has different colors, it has different horn sizes here. Now if we just trained our model just on this image right here and we sit and we pass in this image right here, it's not going to be able to identify that very accurately as a goat because it has never seen an example of a goat that is not looking at us straight on with little horns and ears here. It has never seen this perspective of an image of a goat. So what we need to do is we need to send in a bunch of pictures here so we can start generating a bunch of little maps and connections that represent what we call a goat.

7:36So we would take this picture and this picture and this picture and this picture and this picture and we would label all those, goat, goat, goat, goat, and we would send them into our model and the model will start building similarities and relationships and ratios and begin to build out this map of here's what a goat looks like in many different scenarios. So the more information we send in during the training process, the more accurate our model is going to be at making the decisions that we want it to make.

8:09Let's look at our Lego data set here. If you happen to have looked at the features dot CSV file that came with the set, you'll see that it does have a feature in there. What is a feature? Well, a feature is just an attribute of one of our items here. And one of the features that it's calling out is helmet. Well, there's only one in here that has a helmet and I suppose you could say this is maybe a helmet, but this is not a helmet that's a hood, that's a hood, I don't know what that is. But anyway, we just have one person or one Lego wearing a helmet.

8:42So I didn't really want to just use helmet or no helmet because we don't have a whole lot of data. We don't have a whole lot of variety in our data set. So what I decided to do is I looked at the data set and I see that some of these figures, we can see their teeth, their mouth is open and for others, their mouth is closed. So what I decided to do is let's do a binary classifier here of not mouth open or mouth open. So we have some examples right there, granny with her mouth open versus some of our other characters over here, some sort of empire official right here, I believe, but you can see they do not have their mouth open.

9:23We can use these two sets kind of divided amongst themselves to begin to train our model. We can train our model and send in these pictures and say, hey, these are Legos with their mouth open and these are Legos with their mouth not open. Now what I anticipate our model is going to do is to kind of key off this little white space right here because that is something that is consistent across these different images. And over here, you see no white space right here.

9:54So it's my best guess that the model is probably going to be using that difference to really define whether the mouth is open or not open. Now is that good or bad? Well, we don't know because if we feed in a new Lego with the mouth open and let's say that it just has a black entry in here, like they're screaming or something and they have no teeth visible, well, is that going to rate as a mouth open or a mouth not open? We will have to see that. As I said before, we need a lot of pictures and a lot of samples to send into our model training process.

10:31How many do we need? That's a good question. We can send in fewer images or more images. Now if we have fewer images to work with, we are going to get better results doing something called transfer learning with a pre train model and I'll explain that in a second. If we have a bunch of images, if we're dealing with something is very specialized, maybe we can build our CNN from scratch. We don't have to rely on a pre train model. What is a pre train model? Well, a pre train model is just a model that's just already been trained on some stuff.

11:05In our case, we would probably want to pick a model out there that has already been trained how to look at pictures and there are plenty of open source models out there that we can use as a starting point to get us started and then we can use this transfer learning process. What that just means is that we're going to start with this pre train model and we're going to feed in our own data to make it more specialized on the topic that we 're trying to build our ultimate model to be able to understand.

11:37The difference here is we can either start with dry pasta in a box or if we want to do the artisan path here, we can create our own pasta. We're going to get to the same point. This is going to be a little bit quicker. This is going to take a little bit longer. This is going to be more specialized. Maybe we don't have a pre train model that it's very good for what we're trying to do, but there are lots of pre train models out there for image recognition and image classification. So we are going to pursue this path right here.

0:00Alright, so we're out at our console and our first order of business is to create or recreate our SageMaker AI domain. Now, if you happen to leave yours running, then you can just use it. But I deleted mine and I showed you how to delete yours as well because it's not exactly so straightforward. But now you know how to delete stuff, you can recreate it and just kind of do your own thing. So I'm going to create a domain here and we are going to use the setup for organizations.

0:31As I mentioned before, it allows us to use IAM identity center. If we just use setup for a single user, it only offers IAM authentication, which kind of complicates things because we are on the IAM identity center train, which is where we want to be. So I'm just going to call this SageMaker, come up with your own ultra-creative name. If you would like, we are using identity center here. And we only have one user group in our identity center. That's administrators and I'm going to click on next.

1:03Now, I'm going to go through here and I'm just going to uncheck the S3 bucket access. And I'm going to check S3 full access because I am doing this in a sandbox account, which you should be to. We don't really care very much about access here. We just want full access. If by chance you did not delete the role from last time, if you remember when we created a SageMaker domain last time, we added an additional role there. So if you have already created that role, and this is my role down here that was created, it'll probably say something like SageMaker execution role. Then whatever date you ended up creating it, you can feel free to use that.

1:46Now, if you remember, what we did is we added EC2 full access and we are going to need that access on this role. So you can do this two different ways. If you happen to delete that original role that was created, what I'd ask you to do here is just select full access for S3 and then go back out there and add whatever the name of the role that gets created, go back out there and add EC2 full access. I will show that to you here. So I'm going to IAM. We'll open that in a new tab , and we're going to find that role.

2:22And let's see, what is that role name here? SageMaker execution role with a bunch of numbers. There it is SageMaker execution role. And if you drill into that, if we scroll down here, we see EC2 full access. Now, we're going to need this access when we start working with data Wrangler. So be sure that that is in place. Otherwise, you will get errors and you'll get frustration. I'm going to use my existing role here. That should work fine.

2:54And I'm just going to leave all the defaults here. Click on next. Leave everything on. Click on next. And I want to use a public internet access. Now we talked about this at length last time in the other skill. This means that our resources inside this deployment, this domain deployment will have internet access. There are cases where you do not want that to be the case. In my case, I don't really care. I'm going to choose I'll choose a private subnets here. Private one, two, three , and I believe that privates are recommended because otherwise you may not get some functionality. And security group, we have to choose a secure group. I'm just going to choose the default one because it's pretty wide open.

3:39I don't really have anything that I need to change here. Click on next. And this is just a confirmation screen. And I'm going to click on submit. There we go. Submit. And now I'm just waiting a few seconds until our domain is created. So it's going to take a little while to stand it up and get everything squared away. And then after that, if you remember, what we need to do next is come in here and add a user profile . So I'm just going to pause the video until it is time to do that. All right. So our domain is now in service SageMaker domain. I'm going to go back into it, go to user profile, assign users, and check my little name right there, assign users.

4:20All right. So I have a user profile assigned. So once the user is added there, I'm going to launch an app. I'm going to launch canvas. So this is going to create our application and allow us to launch this in the future. All right. So I launch canvas. Now you may get this message. This is the first time I've ever seen this message, but I'm looking here selected Kendra indexes. We're not using Kendra. So we can probably disregard it may bite me in the future. But anyway, we want to go down here to data Wrangler and make sure we can access it.

4:55You can see the UI is pretty colorful and it's relatively simply laid out. We 're going to revisit this when we start playing with data Wrangler in a few videos. But for now, we want to turn our attention back to SageMaker AI and we're going to create a labeling job.

0:00Okay, so remember back when I was talking about this process where we need several images and we're going to give them names Here of what they are then we're going to send them into our model training process And that is going to help our model develop a better understanding of what a goat is And so to do that we have to number one find a bunch of pictures of goats and number two We have to tell the model that hey this thing that you're looking at right now is a goat And I shouldn't say looking at this thing that you're analyzing and you're stripping down to a bunch of ones and zeroes That representation is something that we refer to as goat So we need to label our data set these are labels Sometimes your data comes pre-label Sometimes your data does not come pre-labeled in our case We have our data set here and we want to divide them into not mouth open and mouth open Well, we could do that pretty easily manually just by looking at each of the 33 images there But no, no, no, that's no fun Let's say that we are dealing with maybe several tens of thousands of images that we have to have somebody look at Because maybe the machine learning capabilities are not capable of determining

1:19whether a mouth is open or not I'm I'm certain that there are models out there that can easily do this But just kind of play along with me. I'm trying to take this on a very basic step-by-step process here So let's say that we have to label these two sets of data Well, we need to do a labeling job and we are going to use SageMaker ground truth to create a labeling job For our data set. So let's head out in the console and do that right now Alright, so our SageMaker domain is working if you were able to successfully start up canvas Then you should be good there we're gonna turn our attention to our ground truth and create a labeling job So we're gonna scroll on here to where it says ground truth and click on labeling jobs So we're being asked to create a labeling job What I want to do is I want to create a job that represents a Labeling activity. So if we go out here to s3 if you remember we uploaded our Lego images out there so let's go in their Lego faces and Had this feature CSV, but we're not gonna use that if we go into this double- zero folder You have all these images right here and those are the ones that I was showing

2:36in the presentation So these are the images that we want to label right now. We don't know what they are We don't know whether they have their mouths open or their mouths shut. So we need to create a labeling job now Of course, we're dealing with kind of small stakes here. We're only dealing with 33 images But just imagine you're dealing with a bunch of images Let's say millions or hundreds of thousands of images and you need all those labels You need somebody to look at those things and say whether or not. Yeah, that's a cat No, that's not a cat or something like that chances are you probably do that with automation without having to put it in front of humans But there are complex complex tasks that sometimes require human intervention We are going to create a labeling job now This job is going to be exceedingly simple But I just want to get you through the process of setting one of these things up So if you need to do this in the future or you see an opportunity to do this you kind of know what you're doing So I'm gonna give this job a name. I'm just gonna call this mouth open or not And we have a couple different options here. We can do automated data setup or manual data setup Now automated data setup is going to read our s3 bucket and it's gonna do us a favor It's gonna create what's called a manifest file

3:52And if we did not do that we would have to manually create a manifest file Basically, this is just an inventory of the images or whatever data we want to have labeled in our case We can do that automatically and it's gonna save us some work so I'm just gonna check that little box right there and check that double zero directory choose and For the output the same location as the input data set. Yep, that's fine The data type is we were talking about images and the I am role I want to create a brand new I am role now I can specify this specific bucket that I want to be able to access or provide access to and that's probably a good thing So I'm gonna go up here Scott Plutcher Lego faces. Let's try this See if that flies There we go and then I'm going to click on complete data setup now what it's trying to do right now is go out there to that directory and Create our manifest file for us. So if we go out there to that directory, let's do a refresh Do we see something in here? There it is right there data manifest So we have a manifest and a JSON file. So that is going to be used when we're

5:08creating our labeling job So now we have our data prepared. The next step is we come down here to the task type And there are several different task types here text video point cloud generative AI Custom all that other stuff we can choose image because that's what we're doing and we are doing an image Classification now you remember we just have a single label here or we could have multi labels in other words This is a human and this is also a vehicle in our case. We're just doing image classification with a single label We're interested in whether or not that image has their mouth open or not So we can scroll down here click on next and now we are asked what type of workers do we want to assign this to?

5:55So this is pretty interesting. So we have mechanical Turk which I mentioned before and that is a human-based Workforce and we can actually send this job out to that mechanical Turk army and then humans would look at our pictures and follow our instructions and decide whether that mouth is open or not or We can do it private which is what we're going to do that just means maybe we have a team ourselves Maybe we've built our own team inside our own department or inside our own company We can do that too or we can do vendor managed Maybe the tasks that we're being asked to do or we're asking our workers to do is very very specialized Maybe it requires specialized engineering knowledge or specialized legal knowledge or something like that Well, we can probably find a company that would provide those specialized skills to be able to properly label our data I'm just going to give this a team name and I'm just going to say my team and Here is how I can invite my private annotators. I can put their email address in here. So I'm going to put my email address Okay, now task time out. This is the mask maximum time that a worker can work

7:10on a single task What this means is that if a worker starts working they're going to give certain answers and then after a period of five minutes they're going to say thank you very much come back later and then it has to time out basically the task has to Reset and then you would come back in there and it'd give you five more minutes to do stuff And the idea here I think the idea here is that you don't want people working on tasks for hours at a time because maybe their accuracy would deteriorate so for our case because I am the only worker and I'm pretty assured that I'm going to be able to get through 33 images and be able to classify them properly So I'm going to change this to 30 minutes And I should be able to do that. Here's some organization information We can enter a description of our organization. We'll just say my org and then contact email Who do they contact if they have questions?

8:08Or issues Chances are you'd probably if you were doing this at scale you'd probably put some sort of group message or some sort of automated Support desk email address here. I mentioned that we can also automate data labeling We could probably easily use this option right here to determine whether the mouth is open or is not open on our Lego Images because I'm sure that SageMaker has a model that has already been at least marginally trained on that task But we are going to do this the old-fashioned way. We are going to do it the manual way So I'm going to keep on scrolling down here. Now here is what the worker is going to see So what it's done is it's pulled a sample image from our collection and we have to enter a brief description here Let's say select Select whether the mouth is open or not open. We will say mouth open and We will say mouth. We'll just say closed. Well, we'll do not mouth not open Because this is a binary classification or either interested in whether the mouth is open or it is not open I could say closed, but that may get a little confusing. So We have this little instruction here. We can add some more instructions over

9:23here I can even preview it but last time I tried this it didn't quite work. I'm not exactly sure why Well, actually it worked this time. That's kind of cool. Here is a preview. Here's what our workers are going to see They can see the instructions You can see we could have added some stuff over here Select whether the mouth is open or not open and we have these two choices here And the worker has to look at this image and say mouth not open and then they would submit it Here's a little submit button. Now this didn't actually do anything This was just a preview and what this is going to send back to ground truth is it's going to send back this record right here It's going to say hey for this picture. The label is mouth not open Let us try this. Let's go ahead and create And it's going to take a little bit of time to create so i'm just going to pause the video until it is done creating this Labeling job. Well, well I won't pause the video because it just created it, but it does take a little bit of time here. Let's see Yeah, I'll just pause the video. Probably in the meantime, you can go check your inbox And ideally you'll have an invitation to be part of this labeling job team. So let me do that All right, so here is what was in my inbox a little invitation. You are being invited Here's a link here. It gives us a username and below this

10:41It gives me a little temporary password that I can use so I am going to click on this link here Okay, so here is the link that I clicked on going to sign in with my email address here And paste in the temporary password and it should prompt me for a new password. I'm going to enter my new password here Here we go Send And get out of here What it is doing is it landing me on this little jobs list here So if I had multiple jobs that I was assigned to if we go back here to our labeling job Go down to ground truth labeling workforces If we go under private, you'll see here is our workforce now We just have one worker on our workforce now you can invite new workers And they would be able to provide input to multiple jobs you can assign multiple jobs And here's the listing here. It says we have a job out here image classification single label And it's available and that's when it was created. So I'm going to click on start working It gives us instructions here and it's asking select whether the mouth is open or not open

11:56Notice the time here. It's given us a little countdown timer and we can either decline this task We can say hey, we don't want to do this. We can release it or we can just skip it I'm going to select not open it submit Mouth is open submit and so I'm going to go through all 33 of these and see how many I can do I'm just going to pause it here and hopefully I can get through all of them before it times me out If I can't get through 33 in 30 minutes, then shame on me So I finished up all the tasks that it gave me now It did not give me all 33 of those it gave me a decent chunk there So I do believe that we have done most of them. Yep. There it is 20 of 33 objects have been classified and that sounds about right it gave me about 20 And if we go into it, we have our labeled objects right here And the labels have been stored or they are being stored out in our output location here. So we can open this up And we can see annotations worker response iteration one And then here's all the the image IDs there and we can see this little file

13:15And let's download that Take a peek Here is the file that I downloaded It's given a lot of information here for that particular image. It's saying mouth not open Gives the timestamp and when it was stored and all that stuff So that is very useful because we can use that in the future to be able to decide whether that image Classified is labeled as mouth not open or mouth open. Let's go ahead and close this Let me go back out to my worker tool here and let's refresh it All right, there we go. So we got another swing in it here We're going to start working and this is probably going to allow us to Finish off the data set. Now sometimes they will give you the same image multiple times And that's just good practice because it wants to get multiple opinions on the label of this So you can kind of triangulate. What's the proper label? So I'm just going to finish off Labeling whatever it's going to give me and we're going to see if that is enough to complete our data set All right, I think that was the last of them it dropped me off in this screen right here. I've refreshed a few times Got no more tasks. I'm going to go back over here to the labeling job

14:31And let's see what we're sitting at Refresh sometimes it takes a little time to update this So i'm just going to pause the video and see if it's going to update that to the number of objects that we had that we're trying to label All right, so it has updated same 32 of 33. I don't know where the last one went But anyway, this is just demonstrating how to use this tool In fact, we are not really going to use this data right now But what I want to do is end this job. So i'm going to go up here to stop That is going to stop the labeling job Basically, it's going to close this opportunity for people to do labeling And just the fact that I closed it it decided to update and give me all 33 So that's what I was hoping to see So just refresh here. It's going to take a little while to stop it But anyway, that's how we can set up a labeling job and then we can use that data to label our images but In our case, i'm going to kind of cheat So I am going to go back out here to lego faces and I have manually labeled these I just basically sorted them So what i'm going to do and you can do the same thing

15:46Let's just presume that we conducted our labeling job and we went out there to the data in here and we looked at all the mouth open and not open and we Use some formulas to assign different values and we ended up with something that looks like this So here we have folders here mouth open folders with mouth not open So I just manually did that i'm going to click on upload and get all that stuff in there So what this is going to do is drop these images into those respective folders there And then that's something that we can use next when we're messing about with data wrangler

0:00In a prior video, I mentioned that we're going to have better success if we get a bunch of images of our classifications. So here we have our classifications, not mouth open, mouth open. But if we go back here to our data set, we only have 33 samples here. And of those 33 samples, there are a few, only a few that actually have their mouth open. So this is not going to be a very useful data set. So what are we to do? Well, one of the things we could do is go out and do a whole bunch of searching and see if we can find more Lego faces here that represent the two classes that we want to classify. Another thing that we can do is something called data augmentation.

0:42So let's say we have our image right here. Well, if we feed this into a model training process, this image is going to be different than this image. It's the same image, of course, but it's just rotated, maybe 30 degrees or something like that. And the training process isn't going to know that this is the same image. In fact, it's going to look at it exactly the same way. It's going to say, okay, what is this? What does this represent? And we can say this is a mouth open image. This is a different image than this, and this, it doesn't have any color, it's black and white.

1:17And our model is going to interpret that with a bunch of ones and zeros and say , hey, okay, I understand that you're telling me that this is a mouth open. So here's what I see significant about this picture. Here's what I see significant about this picture, so forth and so on. This is yet another different image to the model. We've just flipped it and zoomed it in a little bit. And in this image, we put some noise on it, some speckles. And in this image, we've turned up the contrast, made it look a little bit different, flipped it on its side there . So all these images, when it comes to image recognition training, as far as the model is concerned, are brand new images, these are totally separate images. So do you see where I'm going here?

2:00What we can do is use a technique called data augmentation, where we can take our relatively small data set, and then we can run it through some processes such that we can increase the population of training data that we have available to send into our model training process. And therefore, hopefully, our model is going to develop more accuracy. We are going to use data Wrangler to do that process. Next up, we're going to play around data Wrangler and let's get canvas cranking here.

2:35We can open canvas from here. If you didn't see that, I just clicked on canvas. And then it selected the user profile and I clicked on open canvas. And there's canvas. I think we can disregard that Kendra permissions. We're not using Kendra. So I'm going to click on data Wrangler. Now from here, I can import the data. So I'm going to go up here, import image. And then we have different options here. I'm going to import it from s3. And so what I can do is find our data. And I want to import let's work on mouth open first. So I'm going to click on that. And we should have our data being imported right now, no errors, that's a good thing. And so this kind of looks similar to glue or data brew or other ETL tools. But this is our source data. Now I can click on this little tab right here and it's going to show me the data itself. So here's all the pictures. And you can see it has all the mal this open. So what we can do from here is we can add some transformations because our goal here is to augment this data with some different shapes and kind of flavors of data. So we can go back here and click on add a transformation. Let's say

3:53that we want to we'll do a resize or no, we'll do rotate. What we'll do resize first and rotate . So resize crop, how many pixels? I'll say 100 and preview. So let's see what this is going to look like here. Well, it looks like it pushed in a little bit. Let me see if I can resize a little bit more. Let's do 200. See if that looks a bit different. I want something that's kind of markedly different.

4:24There we go. That's better. So basically we just zoomed into the face here. Now according to the model, when we're training the model, this is going to be a totally separate picture, an image, then the other images that we're basing this on. So from here, we can click on add. And if we want to do other transformations, we can do that too. Let's say I can corrupt the image. And I use a noise function there, severity, let's really corrupt it and see what happens here. All right, yeah, that's pretty corrupted there. Add some noise. Sure, we like that. All right, so this is how our data is shaping up right now. So we've zoomed in, we've res ized the images, we corrupted the images. All right, so one of the other things we can do is if we don't find something here that is to our liking, we can create our own, we can come down here and do a custom transform. And here we can actually paste in our own PySpark or pandas or Python code to do whatever we want it to do. But I'm not going to mess with that, get rid of that , get rid of that, let's go back to the data flow here. So what we have is we're resizing the image, we're corrupting the image. Now I can choose to export this data now. And I'm going to export it

5:40, let's say, Lego faces, mouth open, mouth open there, apply, just wanted to get that into this entry here. And I'm going to add another directory underneath that, we're going to say this is resize. Oops, resize noise. So there we go. And I'm going to add an export. And what that's going to do is that's going to basically start this export job process entire data set auto job configuration. Sure.

6:17Now, one of the other things that we could have done, let me show you back here is from here, we can define an export as well. And I think I'll go ahead and do that just for demonstration purposes, choose that, choose that, get that. There we go, apply, get that in there, mouth open, and we'll just say resize. So what this is allowing us to do is create different dropping off points. So we can have a whole cascade of these things. So what this is going to do is this is going to export just resized images. And then this is going to export resized and corrupt images. Now we can add more transformations here as well, and their own respective exports. So we can do this again and again and again. And what that allows us to do is compound the amount of data that we're producing. So forever, every run, we can create more and more data. So we can say, hey, run this once every 15 minutes or so. And then before long, we're going to have a decent stack of data.

7:24But let me go ahead and run this right now. Click on export. And we're going to say process entire data sets. We also want to click on that as well. Because that's going to allow us to export this data set. And then this also as well. And I'm just going to scroll down here, scroll down here, scroll down here, everything else looks good. All right, so our processing job has started. So now what we can do is go check out the job. So to find the job, we have to go back here to SageMaker, and we go down here under processing right here, and we check out processing jobs.

8:02And there's our job. So our job is running, and we're going to let it finish. It usually takes or it probably won't take five minutes or something like that. And then in the next video, we're going to go out and check out what it output it.

0:00Alright, so our processing job is done. Let's hop out to S3 here, see if we can see what it did. Just scroll down to Lego Faces, mouth open, and if we scroll all the way down. Sorry, I have this really large, probably on your screen, the layout's much better. But if we go down here to resize noise and resize, if we go underneath there, we have all our files here, and these are the resized files.

0:31And we should see the same thing here, if we go under resize noise, these are the resized noise edition of the files. One of the things that we would have to deal with here is you can see that it kept the same file name. Ideally, when we're going to be doing this training, we're putting all this stuff in the same folder, and we're going to be pointing our CNN model , our pre-trained model, rather, at that collection of images. But here's the limitation of data Wr angler is that you can see here, it's nice and friendly. But if you're doing this at scale, you're not going to use this UI.

1:09It doesn't really have all the features that you would really want when you're trying to augment and transform your data. Sure, it does transforms here, but it's still pretty limited. What most data scientists will do is they will probably use like a Jupiter notebook or a Python script or something like that. In the future, when the time comes, and we need to augment this data, we are going to use a Jupiter notebook. And we are going to use some special Python functions that are purpose built for augmenting our image data. And they will be able to crank out hundreds of images, really as many images as we need in a few minutes. So I just wanted to take you through this tool just so you know what it's all about, what it can do, kind of what it 's used for, just so you have some sense of when somebody says data Wrangler versus AWS glue versus glue databrew, you have an idea about the differences of those tools. Now let's tear things down.

2:09So really, we only created our domain. So we only have to tear down our domain. Now, one of the things if you remember, is first of all, we have to get rid of the spaces and any applications that are running in that come up here to delete, yes, delete, see if we can delete this. As you saw in the last video, sometimes this gets a little squirrely. And of course, this is no exception here. So I'm going to go back over to my session here, I'm just going to close out, close out of here, we'll go over here to resources, see if we can find that's our canvas running, we're going to stop that. And I think it stopped it, or it is in the process of stopping.

2:49There it is deleting, stopped. All right, now I go back to my space, click here delete. Now hopefully we should be able to delete our space. All right, space was successfully deleted. Let's go over to our user profiles, drill in here, delete our user, delete. And if everything's cool, now we should have an active delete domain button, yes, delete my domain, delete, delete. All right, see you later until next time SageMaker AI domain.

3:31That's really all we have to do. We are of course going to spin up domains again. So we are going to get very good at spinning these things up. The reason I shut them down is because I don't know when the next time you're going to start up the next video. And I certainly don't want you to accidentally leave stuff running that's going to cost you money. So that's why we're deleting things and spinning them up with every unique skill.

0:00Let's take a look at these questions. First question, I'm going to let you read this. Hopefully you already have. Which approach should the engineer take? So let's start down here at the bottom. Use SageMaker Data Wrangler to pre-process the images before labeling. Well, yeah, we could probably do that. But if we're talking about a really large data set, chances are you're probably not going to use Data Wrangler for that. And it doesn't really say what we're going to be doing with this pre-processing. And we're not really asked up here to pre-process anything. Let's try the next one. Deploy a pre-trained model in SageMaker to label the entire data set automatically. Well, we could try that, but this kind of presumes that our data set is easy to label or there's a pre-trained model out there that we can use to label it. So I'm going to put a question mark there. Let's see if there's a better one. Use fully manual labeling with a private workforce to ensure maximum accuracy. Well, yes, we could absolutely do this as well. But one of the things we're asked about here is trying to reduce costs. So this is probably going to be the most costly way of labeling that data. Let's keep on going. Enable automated data labeling with active learning to reduce the number of images requiring human

1:17review. Yes, that is something we can do with Ground Truth. If you remember when we were setting up our Ground Truth job, it had that little checkbox there that we could say, yeah, try some automated labeling. So that is an option to reduce cost. I like that answer best of all. All right, next question. We have a data set, but they have trained a model using that data set and it's underperforming due to limited variation in lighting conditions. And this is not an uncommon problem. Once you train your model, you actually do some testing and you figure out, hey, it's not great in all situations. So what do you have to do? Well, you have to go back to your Ground Truth data and say, hey, do I have to change this or maybe adjust it in some way so that I can give the model that experience that it's not really getting correct? So which data wrangler transform should the engineer apply? Let's try from the bottom here. Remove duplicate images from the data set. No, that's not going to help us one bit. Convert images to grayscale to standardize the colors. Well, you may be thinking, well, yeah, if we just scale everything to gray, then we have no colors and that would be more appropriate. So we could just scale the input image to gray,

2:33but I don't really like this one. This is not probably going to help us with lighting conditions. Adjust the brightness and contrast of the images. Yeah, yeah, that could help if we were to take our existing data set and then add some variation by adjusting the brightness and contrast. Maybe that'll help the model train more accurately for those real world situations where we do have variations in lighting conditions. I'm going to put a little check mark beside this one because this is a pretty strong answer. Launch a project to collect new set of ground truth data. Well, we could do that, but I don't think that's called for. We can try some other things like we can try this adjusting the brightness and contrast before we undertake this new project. Plus it says which data wrangler transform should we use? And this is not a data wrangler transform. Last option, apply text normalization to image metadata. Well, that's not going to help us one bit. It appears we're using this for computer vision, so text normalization is kind of irrelevant. So I am going to choose that one as my final answer. All right, final question. You can read all this stuff, which is the best approach the team can take to address both augmentation and labeling needs. Let's start

3:53from the bottom here. Apply text augmentation and data wrangler to generate synthetic transcripts and use ground truth for audio labeling. Well, that's kind of backwards of what we're being asked here. I suppose we could use text augmentation to generate synthetic transcripts, but audio labeling, what is that going to do? What we're being asked here is transcript verification. So ideally we are sending into ground truth the transcript and then it's supposed to be judged as to whether it's accurate or not. Let's keep on going. Configure ground truth to perform audio augmentation and labeling in a single workflow. Well, ground truth does not really help us do audio augmentation or any sort of data augmentation. It just helps us facilitate data labeling. So we can eliminate this one. This is not a good option. Use ground truth to label the audio clips first. Okay, yeah, we can do that. Then import the labeled data into data wrangler for noise augmentation. Well, I suppose we could do this. Let's keep on going to see if there's a better option here. Use a custom transform in data wrangler to add synthetic background noise to the audio clips

5:10and export the data set to an S3 bucket for ground truth labeling job. Yes, this is probably more accurate or something that I would do rather than this one right here because in data wrangler, although it does not handle synthetic background noise for audio clips, we can absolutely do it with a custom transform and then exporting that out to an S3 bucket. Well, that's the starting point for ground truth labeling jobs. So of all these options here, this is probably my best choice. And remember, maybe these aren't all perfect options, but it says here the best, and I would say that's probably the best of these other options. So I hope this has been informative for you, and I'd like to thank you for viewing.