How to use TargetCreateTargetRequest class of PuppeteerSharp.Messaging package

There are few ways to implement what you need (considering a C# console application).

Maybe the easiest one is to use tools that interact with an instance of a browser, i.e. Selenium (used for unit tests). So:

1. Install Selenium.WebDriver nuget package
2. Install a browser where your application will run (let's suppose chrome)
3. Download the browser driver (chromedriver)

4. Write something like:

   IWebDriver driver = null;
   try
   {
       ChromeOptions options = new ChromeOptions();
       options.AddArguments("--incognito");
   
       driver = new ChromeDriver(options);
       driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(5);
       driver.Url = "https://www.wedj.com/dj-photo-video.nsf/firstdance.html";
   
       var musicTable = driver.FindElement(By.Id("musicTable"));
       // interact with driver to get data from the page.
    }
    finally
    {
       if (driver != null)
          driver.Dispose();
    }
   

Otherwise, you need to investigate a little bit more how the webpage works. As far as I can see, the page loads a javascript, https://www.wedj.com/dj-photo-video.nsf/musiclist.js, that it is responsible to load the list of music from server. This js script basically load data from following url: https://www.wedj.com/gbmusic.nsf/musicList?open&wedj=1&list=category_firstdance&count=100 (you can open it also in a browser). Excluding "(" and ")", the result is a json you can parse (maybe using newtonsoft.json package):

{
  "more": "yes",
  "title": "<h1>Most Requested Wedding First Dance Songs<\/h...",
  "event": "<table class='musicTable g6-table-all g6-small' id='musicTable' borde..."
}

The event property contains the data you need (you can use HtmlAgilityPack nuget package to parse it).

Pro Selenium:

1. easy to interact with
2. the behavior is the same of what you see by the browser

Cons Selenium:

1. you need chrome or another browser installed
2. the browser is running when you interact with it
3. the browser download the full page (images, html, js, css...)

Pro manual:

1. you load only what you need
2. no dependencies to external programs (i.e. browsers)

Cons manual:

1. you need to understand how html/js works
2. you need to manually parse the json/html

In this specific case, I prefer the second options.

You can use the Response event:

var page = await browser.NewPageAsync();
page.Response += (sender, e) => 
{
    Console.WriteLine(e.Response.Url);
}

You can use different browser instances for each logical instances. I mean instead of trying to set different proxy for each page/tab with different proxy just create new browser instance and set proxy via launch args.

If this solution doesn't fit your needs, check this question. There is library for NodeJS which give ability to use different proxy per each page/tab. You can check that library source code and implement same things inside your C# application.

That library is using very simple method. Instead of sending requests via puppeter's browser/page library send request via nodejs http tools. It can be done by using method page.setRequestInterception. So library intercept each request from page, after that gather data and send request via http tools. I used C# a long time ago. So maybe I am wrong, but you can try to use HttpWebRequest or something similar. After you get result you should use method request.respond and pass response results there. In this way you can put any kind of proxy inside your application. Check here code of library.

If you are using CSS selector as a locator then you haven't provided correct locator.

You can try following CSS Selector which should work.

Using Type attribute

await page.ClickAsync("button[type='submit']");

OR Using name attribute

 await page.ClickAsync("button[name='login-submit']");

OR using class attribute

await page.ClickAsync("button.inline-btn-2");

OR Using multiple attribute such as

await page.ClickAsync("button[type='submit'][name='login-submit']");

The content you trying to scrape here is inside iframe.
To access it you have to switch to that iframe.
See here example how to switch to iframe with BeautifulSoup.
Here example with Selenium scraping inside iframe.
And here with puppeteersharp

Thanks all for your responses. I used HtmlRenderer external dll (library) to achieve the same and found below code for the same.

Here is the code for this

public void ConvertHtmlToImage()
{
   Bitmap m_Bitmap = new Bitmap(400, 600);
   PointF point = new PointF(0, 0);
   SizeF maxSize = new System.Drawing.SizeF(500, 500);
   HtmlRenderer.HtmlRender.Render(Graphics.FromImage(m_Bitmap),
                                           "<html><body><p>This is some html code</p>"
                                           + "<p>This is another html line</p></body>",
                                            point, maxSize);

   m_Bitmap.Save(@"C:\Test.png", ImageFormat.Png);
}

This is how you can use System.Net.WebSockets.Managed on Puppeteer-Sharp

var browserOptions = new LaunchOptions
{
    Headless = true,
    WebSocketFactory = (uri, socketOptions, cancellationToken)
        => System.Net.WebSockets.SystemClientWebSocket.ConnectAsync(uri, cancellationToken)
};
using (var browser = await Puppeteer.LaunchAsync(browserOptions))
using (var page = await browser.NewPageAsync())
{

}

The following code was working fine:

await page.WaitForSelectorAsync(".event"); // wait ajax completion
var divs = await page.QuerySelectorAllAsync(".event__header--no-my-games > .event__info");

foreach(var cd in divs)
{
   await cd.ClickAsync();
}

the problem was that the browser opened by Puppeteer had mobile dimension, in that case the selectors that I was looking for was differents.

I fixed that declaring a Viewport size in the following way:

Browser = await Puppeteer.LaunchAsync(new LaunchOptions
{
      Headless = true,
      ExecutablePath = "your chrome path",
      DefaultViewport = new ViewPortOptions { Height = 1280, Width = 800}
});

this will open the site in desktop mode with correct selectors.

Hope this help someone.

You can close the page using CloseAsync:

var page = browser.NewPageAsync();
////
await page.CloseAsync();

An using block will also close the page:

using (var page = await new browser.PageAsync())
{
///
}

Puppeteer-Sharp v2.0.3+ also supports await using blocks

await using (var page = await new browser.PageAsync())
{
 ///
}