Scrapy code for scraping several e-commerce website

I want the product name/title, unique product code, current price, availability, discount (if there is any), size and fit information, list of product codes with alternative colors, and all original image urls associated with the product (typically 4-6 image urls per product). Original image urls meaning that it is not the thumbnail version since that is too small to view. I describe where they are in the first picture attached, which is a clothing: 1) product name/title: in this case, Ophelia Wide Leg Flare Jeans 2) For this website, unique product code is farfetch id, which is at the center bottom. 3) Discount and price info is at top right 4) there are four images associated with this dress, which you could click to check them all in the top left. Somehow this sight does not have thumbnail version, which is good. 5) list of product codes with alternative colors： does not apply for clothes usually, therefore an empty list [] 6) Fit and Size information is a bit complicated. There are two parts and I want both. FIrst is in the section Size & Fit, here for clothes, it is generally the measurement of the model and which size the model is wearing. That is fine, I just need the entire text. The second part is the dropdown menu on the top right: "Select Size". I need to get all the available sizes, in this case ["23 WAIST", "24 WAIST", ....] AND there availability and price for each size. Availability are very often different for different sizes, as we can see here 23 and 24 are "Last 1 left" while the rest shows nothing, which means "in stock" in general. Also different sizes sometimes have different prices, I also want to get that information. So to sum up, for the first part, I want a big text string containing whatever is in Size & Fit tab. On the second part, I want in the select size dropdown menu, a list of tuple (size, price, avilability) per size option. Can use NA if nothing is there. Point 3) above is mainly relevant for bags, for example, in the attached second picture, which is a brown handbag: on the top right just under "Add to bag" there is a button for a red bag where you could click to get into another bag with exactly the same style but just different colors. I want that red bag's product ID to be stored here with the current brown bag as list of product codes with alternative colors (and by symmetry, current product code should also be stored as alternative color product code list of the red bag). I need to a list here since sometimes there are 4 different colors for a bag. For clothes, this could just be an empty list []. I want it done for two categories: women clothing and women bags, on U.S. store of the following websites: [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] Some notes: 1. I did some easy check, and found that one website [login to view URL] needs to get around reCaptcha. I have no idea whether it is possible to do or not and how hard it is. If it takes too much time, then forget it since I just want to get some data and it is OK to lose one website.. 2. The approach I have in mind is to scrape start from, say for handbag, page shop all bags. And scrape all bags on the first page and move on to the next, etc.. For Neyman marcus, the page is: [login to view URL] There we could see the importance of doing step 3) above. If we just scrape everything page by page from [login to view URL] and do not check the alternative colors, we will lose those bags, since if a bag has 4 colors, only one will show up on all handbags page and in order to get everything it is crucial to check for alternative colors.