Welcome to this hands-on AI-900 lab session, where we explore how to use Azure AI Document Intelligence Service (formerly Form Recognizer) to extract, analyze, and process data from documents using custom forms! Whether you're preparing for the Microsoft AI-900 Certification or looking to automate document processing with AI, this step-by-step tutorial will walk you through everything you need to know.
🔍 What You’ll Learn in This Video:
1️⃣ Introduction to Azure AI Document Intelligence Service 📑
2️⃣ Setting up Document Intelligence in the Azure portal
3️⃣ Training a custom model to recognize structured & unstructured documents
4️⃣ Extracting key-value pairs, tables, and handwritten text
5️⃣ Real-world applications for automating invoices, receipts, and business forms
🛠️ Who Is This For?
Beginners exploring AI-powered document processing
Professionals preparing for the Microsoft AI-900 Certification
Developers & businesses looking to automate form data extraction
📌 Key Highlights:
✅ No-code/low-code setup for intelligent document processing
✅ Custom model training to extract specific data from unique forms
✅ Automating workflows for businesses & enterprise applications
✅ Step-by-step hands-on demo using Azure AI Document Intelligence
Explore Our Other Azure Courses and Practice Material On: https://www.youtube.com/@skilltechclub
🔍 What You’ll Learn in This Video:
1️⃣ Introduction to Azure AI Document Intelligence Service 📑
2️⃣ Setting up Document Intelligence in the Azure portal
3️⃣ Training a custom model to recognize structured & unstructured documents
4️⃣ Extracting key-value pairs, tables, and handwritten text
5️⃣ Real-world applications for automating invoices, receipts, and business forms
🛠️ Who Is This For?
Beginners exploring AI-powered document processing
Professionals preparing for the Microsoft AI-900 Certification
Developers & businesses looking to automate form data extraction
📌 Key Highlights:
✅ No-code/low-code setup for intelligent document processing
✅ Custom model training to extract specific data from unique forms
✅ Automating workflows for businesses & enterprise applications
✅ Step-by-step hands-on demo using Azure AI Document Intelligence
Explore Our Other Azure Courses and Practice Material On: https://www.youtube.com/@skilltechclub
Category
🤖
TechTranscript
00:00hello friends good morning good afternoon and good evening from whichever part of the world
00:14you're watching this video i would like to thank you for joining this particular learning my name
00:19is maruti and i'm back with another azure ai service and today also we are going to focus on
00:25step-by-step practical demo of this particular service the name of the service which we are
00:30going to use today is azure ai document intelligence service now if you do not know what this service is
00:36i strongly recommend you to check this particular video in which i'm explaining the concept of
00:41document intelligence service and right now if i just have to give a brief about this particular
00:46topic then let me tell you this is one service which is actually going to be used by organizations
00:52and enterprises to extract information from their physical documents we are living in the world
00:58where we have a huge amount of data mostly in a structured unstructured semi-structured way
01:04now think about the scenario when you have an organization with digital documents as well as
01:09there are so many physical documents are also there document intelligence service is going to help you
01:14to capture that information which is stored in those physical documents now obviously this is an ai
01:20based service so somewhere machine learning is going to help you in order to achieve all these things
01:25how exactly you're going to use this how exactly this customized model training with your
01:30organization specific forms is going to work that is what exactly we are going to see in today's video
01:36so do not miss this this is very important so i strongly recommend you to go through this video step by step
01:42also if you have your own azure subscription i strongly recommend you to try this thing with this
01:48all the documents all the files which are required for this lab will be available in the description
01:53of this particular video let's get started i am in my azure portal and as usual the first thing
02:00which i'm going to create is i'm going to create my azure service so i'm going to click on ai plus
02:05machine learning and the service which i want to create is a document intelligence service remember
02:10the older name of the service was form recognizer but the new name is azure ai document intelligence
02:16service which sounds very good i'm going to click on create specifying certain details like my
02:23resource book name my subscription name i'm giving a name ai 900 rg because this is also one of the
02:30service which is part of the ai fundamental course i'm going to select my region which is going to be
02:37east us 2 for this one and the name of the service i'm giving maruti doc intelligence svc
02:452025 because this is 2025 in the pricing tier i'm going to choose standard well for this demo
02:53free is also going to work but i want to try some customized things so that's the reason i'm going
02:58to use standard as zero and within a minute it's going to create our document intelligence service
03:07validation passed i'm going to click on create yes deployment is submitted all the documents all the
03:14files which are required for this lab will be available in the description of this particular
03:18video let's get started i am in my azure portal and as usual the first thing which i'm going to create
03:25is i'm going to create my azure service so i'm going to click on ai plus machine learning and the
03:30service which i want to create is a document intelligence service remember the older name of
03:35the service was form recognizer but the new name is azure ai document intelligence service which sounds very
03:41good i'm going to click on create specifying certain details like my resource group name my subscription
03:48name i'm giving a name ai 900 rg because this is also one of the service which is part of the ai
03:56fundamental course uh i'm going to select my region which is going to be east us 2 for this one
04:03and the name of the service i'm giving maruti doc intelligence svc
04:102025 because this is 2025. in the pricing tier i'm going to choose standard well for this demo free
04:18is also going to work but i want to try some customized things so that's the reason i'm going
04:21to use standard as zero we'll click on review plus create i think everything else is fine and within
04:28a minute it's going to create our document intelligence studio
04:34and within a minute it's going to create our document intelligence service
04:40validation passed i'm going to click on create
04:43yes deployment is submitted now while the deployment is going on i should remind you one thing
04:47that if you're liking our videos if you're interested in azure ai
04:51i strongly recommend you to please like this video and subscribe to our channel
04:56you subscribing to our channel is very important for us
04:58yes deployment is completed let's click on go to resource and just make sure that if the service
05:05is successfully created you have a valid key and endpoint available in that so yes service is active
05:13if i click on keys and endpoint it's going to show me that i have a key and endpoint available which
05:18means that service is successfully created and this is the time we have to use the service by going into
05:24our document intelligence studio now when i click on this link if there are chances that it's going
05:29to open the studio and it's going to link my service with that automatically if you are not
05:34getting automatically linked with this you have to sign in like this is showing me a section here
05:39where i can click on the small icon on the right top corner i'll click on sign in i'll sign in with my
05:45same account in which i have created the document intelligence service
05:50choosing my account yes i have successfully signed in now let me just check the settings part of this so
05:56that i can see whether the resource is successfully selected or not it is very important that you choose
06:02a correct resource while using a document intelligence studio so now i'm the resources tab and it's showing
06:09me that the resource which i've created just before few minutes is actually available i'm going to
06:13select this and i'm going to say use this resource they're asking me are you sure you want to switch
06:18to your default resource i'm saying yes and when i do this thing now onwards this resource is going to
06:23be selected with my subscription with my resource group details in this particular document intelligence
06:29studio i'll click on the document intelligence studio icon now and now it's showing me different kind
06:34of templates which are available you can see we have some very basic templates for document analysis
06:40and layout and general document analysis kind of thing we have some pre-built models inside this
06:45which are perfect for taking and extracting information or analyzing the documents which are invoices
06:52receipts identity documents and you also have certain tech specific forms like you have us personal
06:59tax mortgage pay stubs bank statements you can actually have this kind of pre-built documents
07:06available with this pre-built models and then you can actually use any of them without even model
07:11training you can extract information from them now right now we are first going to try the layout one
07:19so that you can understand how exactly this document analysis is going to work now you can see in this case
07:24this is showing me three sections it's showing me a run analysis button we have a query field if i
07:30want to extract additional fields from your document and we have a setup and analyze option configuration
07:35here as of now i'm going to skip all these things by default it's showing me one particular form which
07:41is a generalized document it's not having any specific formats here so if you have a document which is
07:47not having a static format you can actually put it here and you can run analysis on this this run
07:52analysis is going to extract the information so basically it's going to read the text from this
07:57document and not only that it's actually going to show you segregation of this page content
08:02properly like they are knowing that this is page title this is the title of this thing this is section
08:07heading this is paragraph they are actually able to understand each and every character as well as
08:12there are some diagrams there are some charts they're able to understand all those things
08:17if you check selection marker tables figures whichever is added inside this
08:22this result is going to be available in the json format and this full analysis is going to be
08:26done within few seconds now as of now i do not want to go with the very generalized documents
08:32so let me be specific and let me click on some of the receipts i can click on try it out
08:38it's showing a similar kind of page and from the left side option i can choose any receipt
08:43the moment i choose the receipt and if i say that i want to use this receipt for analysis i can click on
08:49run analysis again it's going to analyze this thing it's going to extract the text information
08:54from this and remember it also knows which fields are this so basically if this is a recept the word
09:00contoso which is written on that particular receipt they know this very well that this is not a normal
09:05text it's a merchant name and you can see when i move my mouse on this it's showing me it's a merchant name
09:10this is merchant address this one is merchant phone number this is transaction date this is time
09:17this is something which is not detected here so if this kind of things are there you have to
09:21configure that thing association with that now it's also knowing that this is a total price this is a
09:27description of the product and maybe this is something which is a quantity which they have
09:31associated with this they are showing me total they are showing me text associated with that and the
09:36total which is included with the text now all these values are extracted and these are actually
09:42extracted with the help of the pre-built models because we have a pre-built models for receipts
09:48now if you have certain receipts which are not working with this one you can actually create a
09:53custom model also or maybe assume the scenario assume in your organization you have some specific
09:59forms which are not exactly matching with some pre-built models you can actually create your custom
10:04models but the rule is you need to have at least five sample forms and if you provide five sample
10:10form inputs you can actually train that model which will be a custom model and using that you can
10:15extract information for your organization document also and this is a very cool feature for most of
10:20the organizations when they want to extract information from their physical documents all they
10:25have to do is they just have to scan them they have to associate with the service and all the information
10:30will be converted into json with the proper field and then they can also utilize that as a knowledge base
10:37where they can actually search that particular content and they can get the desired result from
10:42that now let's try that one so i'm going back to my document intelligence studio and if i scroll little
10:48down from this pre-built models i will get a section which is custom model now in this also we have
10:54custom extraction model to extract the information from your forms and we also have custom classification
11:01model this is even little more complex scenario now today we are going to try custom extraction
11:06model but let me tell you custom classification is going to help you to do classification of various
11:12forms assume you do not have only one type of form in your organization you have multiple types of forms
11:17in this case you can actually put all of them into this and then it will classify which of the documents
11:23are maybe a contract which are assigned with the clients which of the documents are resume which of
11:28the documents are some kind of a salary slip or some other documents within the organization
11:32it will do the classification with that particular service now we are not going to learn classification
11:38right now but if you want a custom classification model specific practical lab you can just comment
11:44down in this particular video and i'll surely create one lab video for custom classification of the
11:49model right now let me click on custom extraction model i'm clicking on get started it's taking me here
11:58and you can see right now i have an option here where i can create a new project so basically
12:02this is going to be your first project which you are actually going to create in document intelligence
12:07service all others were demo we just tried it and then it was working fine i'm giving
12:12some kind of a project name let's say i'm giving this is uh maruti custom extraction
12:20or i can say form extraction or i can say form extraction and this is fine in this case i'm
12:27going to say my custom forms will be used and i'm going to click on continue it's asking me which
12:37subscription you want to select i'm choosing my subscription i'm choosing my resource group
12:42and then i am choosing my document intelligence service now the api version which they are showing
12:51me here is available with the journal availability i always advise you use the latest version of this
12:57service which is available in this list i'm choosing the one which is launched somewhere in the november
13:022024 we are okay with this i'll click on continue and once this is there this option is very important
13:10this is going to ask you connect to the training data source now where is your training data is very
13:14important in this when i'm going to choose this thing i actually need a storage account i need a
13:19blob storage kind of a data so that i can have my training data inside this we will set up that but
13:25right now if i choose my subscription and then if i choose my resource group i do not have any storage
13:31account in this so it's showing me no storage account available i can tick mark here and i can say create
13:37a new storage account so this service is actually going to create a new storage account also for me
13:42i'm giving a name of this thing this is maruti data storage with some number at the end of this
13:51location for this i want same east us2 pricing tab i think we are okay with standard lrs as of now
13:59and blob container name is going to be sample forms the folder path i don't want to give so this is
14:07fine i'm going to click on continue review and create create project remember while the project is
14:14getting created understand we are creating two things we are creating custom extraction model based
14:19project in document intelligence service and internally this is going to create one storage account also
14:24with one empty container once this process is done we'll go back to the storage account we'll see
14:30that what kind of container is actually associated with that and then we are going to use that thing
14:35when we configure our custom model extraction now this project is successfully created before i go to
14:42my azure portal and show you the storage account i wanted to see this particular section is showing you
14:48please upload at least five documents for labeling and training custom extraction model now this is
14:54something which is a part of this and as i explained five minimum documents are required also when i click
15:01on next it's showing me add a field so you can create a field for data labeling options for a field selection
15:08mark signature and table all those things you have to do inside this using add a field you can actually
15:14configure your fields with that we'll do this thing very soon but before that let me just click on
15:21my azure portal let me go to ai 900 rg and do we have a storage account yes we have a storage account
15:30we have a container inside the data storage we have containers and inside the containers we have sample
15:41forms which is obviously going to be empty right now so we do not have any kind of forms inside this
15:46it's empty now let me go back to my document intelligence studio and what i'm going to do is
15:52i'm going to browse for files i have some sample forms available in my computer so i'll go to
16:00a specific folder i have form one two three four five i'm going to open them
16:07it's showing me that do you want to start labeling now uh or you want to do some run layout kind of
16:15automation with that or you want to manually do that thing all these options are available
16:19now i'm going to say i do not want to do it right now i'll skip it now you can see i have form one
16:24form two form three form four and form five each form is actually having same kind of a format but we
16:32don't know what kind of fields are actually there now there is a mechanism which can actually do auto
16:37labeling kind of thing for me and then i can customize that but let's say first i'm going to
16:41click on form one okay now i'm going to just zoom in this form a little bit and once i can read this
16:50thing i'm going to add certain fields like for this hero limited i'm going to say i want to add one field
16:56which is company name so this is going to be a company name field i'll add one more field which is
17:03going to be for company phone website and email so i have to add three fields
17:10and then maybe same way i'm going to specify po date and po number
17:40after this i can take the vendor name and other details also but i'm not going with all those
17:50things right now it's going to take a lot of time and then i'm going to specify things like
17:56details is actually talking about what kind of products we are dealing with so this is going to
18:00be product quantity unit price and total so let me add four fields for that product
18:10qty for quantity
18:15unit price
18:19and total
18:22now i have this 10 different fields which i have created for my form same way you can use
18:28multiple things you can use for subtotal text total other things also as many fields whichever data is
18:34visible on the form you can create that many fields now after creating field you can actually choose the
18:39proper data type also for that so you can see for each field we have a subtype obviously company name
18:43will be a string so i'm okay with that but maybe company phone number it's not going to be a string
18:49so i can say this is going to be some kind of a number and then i'm going to specify it's not a
18:54currency so we are not going to specify anything in that website email is fine po date is going to be of
19:00type date and then in that also we can specify what kind of a format is there like my purchase order date
19:07is actually having month date and year so it's going to be m d y then i have products and product is just
19:16going to be string so it's fine quantity and unit price now in the quantity i'm going to say it's going to
19:23have number unit price is going to be number but this is going to be some kind of a currency i'm going
19:32to say unit price is going to be currency and total is also going to be currency now you can specify
19:39this kind of beta type for your field whichever is matching with that we can do all this thing for
19:44multiple fields actually but right now i'm just showing you this as a demo so i'm not doing it for
19:48other data which is available now once you have created field once you have given the data types
19:53all you have to do is run layout when you run layout they will try to analyze this particular form
19:58which is a custom one so we'll try to run layout now you can see they have highlighted all the content
20:03of this one i can click on the each item now so let's say i'm clicking on hero limited
20:08so hero limited is selected now and i'm saying this is company name so this is going to be associated
20:14with this then i can say company phone so this is actually company phone this is a company website
20:25same way this one is email
20:30this is po date
20:34this is your number
20:36then i'm selecting this one which is products this is quantity
20:51unit price
20:54and total now once you do this it's going to understand that okay these are the fields which
20:59are actually having this kind of values associated with that same way the same process you have to
21:05actually repeat on the other forms also so i can click on this auto label but auto label is a
21:12simple process which is helping you with the pre-built models so right now if i click on this auto label
21:17it's going to ask me that okay which model id you want to select in this now i do not have any particular
21:23pre-built model for this kind of a form that's why i'm not doing this thing so creating a custom model
21:28training is something which is little time consuming as well as painful because you have to label all
21:33these things because this is a custom model and you only know which label it means what
21:37but if you're using pre-built model then all the things will be done automatically so remember that
21:41and keep that thing in mind now i am going to do the same thing for the remaining forms so i'm just
21:47going to fast forward this thing right now because there is no point of doing the same thing for
21:51other forms let me do it and then after this we're going to go to the next step okay as you can see
21:59i have done this kind of labeling with all the fields in my all five forms now at least five forms
22:04are required to train our model and that's the reason we have done this thing once your labeling
22:09is actually done you have to do the next step which is on the right top corner you have a train button
22:14i'm going to click on this train button it's asking me what would be your model id
22:18so let's say i'm going to enter some kind of a unique model name so this is my custom
22:22PO model model description this is used in hero company whatever build model now in this case we
22:38actually have an option you can go with the neural model which is recommended that is something which
22:43is a very good model but it's going to take a lot of time actually and cost also so i'm going to choose
22:48template model template model is something which is easier and faster than this one
22:53i'm going to click on template i'm going to click on train
23:00now it's showing me training is in progress and they are saying your custom PO model is now training
23:06so you can check the status by going to model section i'm just clicking on go to models which
23:11will take me to this particular page which is models now let me tell you one thing this particular
23:16model training can take little longer depends upon the forms you have submitted and depends upon
23:22the model template which you have selected with that now as of now this is showing me this
23:26particular model is there the status of this model is succeeded so if i go inside this model
23:32it's showing me that your build model is template the accuracy for all the things is actually given
23:37for some of this thing the accuracy is very less because i have intentionally selected some wrong fields in
23:43that so normally products quantity unit price total this are having mixed kind of variations inside that
23:49in my form selection which i've done that's the reason it's having less accuracy which is obviously
23:54possible but this model is successfully trained it's showing me that your model is successfully trained
23:59within this short period of time and then obviously right now it's not going to be performing very
24:04well but if i go to the taste section this is a place where i can actually browse for a file and
24:10then i can put it here and it's going to highlight that so let's say i'm going to browse for a file
24:15i'm going to choose one of the same kind of a form only logically you have to put some other form
24:19so i think i have one form also which is taste one i'm going to open that this is for tasting kind of
24:25a purpose i'm loading this thing and now i'm going to see that i'm going to run analysis and let's see
24:31that if this is actually capable to analyze all the fee successfully or not if this is able to do
24:36this thing then this is a good model and i can use it if not i have to retrain that model with more
24:41data now i just did this thing it's showing me that this is a field which is company name these are
24:47the fields company phone website email purchase po date po number these things are not identified because
24:54i have not used those fields in the model training but they know that this is actually something which
24:59is a list of products it's knowing that this is quantity this is unit price and this is a total
25:05so they actually understood this thing very well and that's what the goal of machine learning model
25:10training now this particular video is actually showing you how you can customizely extract data
25:17from your custom forms in most organization this is very important thing but remember job is not done
25:23here this kind of hundreds and thousands of forms can be there in your organization when you are
25:28extracting this information from this particular model remember the result of this particular analysis
25:34is always going to be in the json format extracting data is one step only second step is going to be
25:42making sure that you are able to get the business insight from this particular data and in those kind of
25:47things you have to make sure that you're going to have ai enrich pipelines associated with this data
25:54now i'm just trying to introduce another ai service which is known as azure ai search service
26:00that is actually going to help you to search the proper data from this kind of adjacent content
26:05we are going to create some of the videos on azure ai search service which is basically the next step
26:10once you create this kind of form data extraction because this kind of hundreds and thousands of form data
26:17will be stored into some kind of a knowledge base and then with the help of an indexer process we are going to
26:23create azure ai search service which will help you to search a specific content from this particular data
26:30thank you so much i'll see you tomorrow happy learning