how to put robots.txt on website shopify

hey welcome to another video in this,video i am going to show you,how you can edit robots.txt file in a,shopify store,and robot.txt file is not something you,often change but there are certain times,where you want to add,a file to it or you want to exclude,something from,search engines that you don't want to,crawl so if you don't know what is the,robust.txt you might want to learn it,because in the future you might need it,so robot that txt is a file every,website have,it basically tell the search engine,which page to crawl and which page,not to crawl um the thing is like in,shopify it was not possible,in the last few years and a lot of,people was asking about this one and it,was not possible,so i found out like they have added this,new feature now you can edit,the robots.txt and that's like an,amazing feature so,i thought maybe i can share the videos,so let's see how you can edit this one,now before we do it i'm going to show,you how you can access the robots.txt,in this example team that which i have,it is completely a custom theme,i'm going to go to the url and slash and,then from here,you can see robots.txt and press enter,this is the content of the robots.txt,for example,you can tell the user agent which store,means,everything and like google yahoo,whatever search engine or any,any crawler who want to crawl your,website either they are from coming from,hdf,or they are coming from i don't know,some other crawlers,you can disable the crawler from there,for example they cannot check the,download directory here anything which,is in the download directory,they cannot crawl that because if it is,in the download,directory like they can crawl and people,can google it and they can find it as,well as the admin card order checkout,all of these are disabled,so this is how you can disable it for,example the search is also disabled,because in the search page it is always,custom people search for the custom word,it is not a page where you have to see,it on a google search engine,so how you are going to modify this one,according to the documentation here,which is i think a new one,you just have to create a,robots.txt.liquid file in your,template directory so here i will bring,up my team here,and i make sure it is visible to,everyone,and i'm going to run the theme watch,allow live,and now it is going to watch for any,changes and my live directory,and i'll go here this is the file that,we have to create,i'm going to copy the name from here and,then we come here in the template,directory,let's create it make sure your file does,not have any space,so i'm going to remove this spacing and,press enter,it did create the file and now let's,check the robot.txt now if i refresh it,it looks still the same because you,cannot modify this,data which you have here if you want to,uh modify this one,you can add things to it but you cannot,remove it because,uh it is a risk if you remove download,from here,so how you are going to modify this one,just check out one of the examples,now if you want to search like learn,more about it you can read in the,documentation,it basically give an example of what a,user agent is,robots is an object which have like,different group if i give you this,example and i,added in my file here,it is going to see if user agent value,is equal to,star let's check out our code here,which refer to this one user agent equal,to star it is going to add a certain,value here,so the values disable i start anything,which is start with a queue,currently we don't have this one to,prove you okay just,like you should know it is not there and,for now i will save it now let's come,here,refresh it once you refresh it you can,see it appearing here,now anything which is after this query,which is similar to the search page,and so far you can ignore this when you,can disable it,to all crawlers for example you can add,another example,if i just duplicate this line and say,anything,which is okay you can write a regular,expression here,but i'm going to say private mode,which is a private page maybe it is it,is just for demo,nothing like private exists here i,refresh it and you can see,this is the private uh mode which,appear here and you can search for any,other user agent also like google,artsbot you can search for any other,thing you can add your own if you want,so here is a like a full uh example of,how you can,like remove the default one so you can,come here and,look through them and you can search for,any rule and if you,see the rule that the like,redirect is equal to disallow you can,change it to policy whatever you want to,do,you have the flexibility to do this one,and,also you can add things to the cyborg,the last example that they give you is,you can add url to your,for my website this is the sidemap and,if i check it this is how much uh uh,like links i have if you don't know what,is sidemap it is basically,what you see on the google page for,example if you search for any brand,in the homepage of the google they are,going to show you some,links as a sub category and those are,all,the top cut like links which every brand,have,so if i come here you can add the,sidemap here,with your own sitemap url if you have,you might have a,separate sitemap url so you can use that,one,and that's basically it um i hope it has,been informative,and really great feature by shopify i,know it took a long time but,they did it finally so however it has,been informative thank you for watching,and i will see you in the next video

Congratulation! You bave finally finished reading how to put robots.txt on website shopify and believe you bave enougb understending how to put robots.txt on website shopify

Come on and read the rest of the article!