Switchbot API Scrapping

[Powers]

Hello All,

I have been working to get scraping to work for a while and hit a brick wall and wanted to check if I am the only one or this could be a bug.

I am currently using VC version 2.3.0.4, however, also tested on version 2.3.0.8 with same results.

I am trying to get a list of scenes from a switchbot api (https://github.com/OpenWonderLabs/SwitchBotAPI) using the VC scrape action.

I have the following code:

Code: [Select]

<?xml version="1.0" encoding="utf-16"?>
<!--VoxCommando 2.3.0.4-->
<command id="-1" name="Actions copied from LCB">
  <action>
    <cmdType>Scrape.Get</cmdType>
    <params>
      <param>https://api.switch-bot.com/v1.0/scenes</param>
      <param>application/json; charset=utf8</param>
      <param>switchbot</param>
      <param />
      <param />
      <param>Authorization: <api key></param>
    </params>
    <cmdRepeat>1</cmdRepeat>
  </action>
</command>
The logs show the following:
2/9/2021 8:28:55 PM   721   doCommand:Get Scenes
2/9/2021 8:28:55 PM   721   Action claims to be threadsafe.
2/9/2021 8:28:55 PM   721   action repeat set to: 1
2/9/2021 8:28:55 PM   721   Action:  Scrape.Get
2/9/2021 8:28:55 PM   721   Param1:https://api.switch-bot.com/v1.0/scenes
2/9/2021 8:28:55 PM   721   Param2:application/json; charset=utf8
2/9/2021 8:28:55 PM   721   Param3:switchbot
2/9/2021 8:28:55 PM   721   Param4:
2/9/2021 8:28:55 PM   721   Param5:
2/9/2021 8:28:55 PM   721   Param6:Authorization: <api key>
2/9/2021 8:28:55 PM   721   addHistoryItem [action] Scrape.Get:https://api.switch-bot.com/v1.0/scenes&&application/json; charset=utf8&&switchbot&&&&&&Authorization: <api key>

2/9/2021 8:28:55 PM   736   Doing scrape put/post with headers:
Content-Type: application/json; charset=utf8
User-Agent: switchbot
Authorization: <api key>
2/9/2021 8:28:56 PM   2   Action ERROR: Web exception: The underlying connection was closed: An unexpected error occurred on a send.
The underlying connection was closed: An unexpected error occurred on a send.
 
Unsure why I get this error. I verified in a Linux machine with a curl command and it works just fine:

Code: [Select]

curl https://api.switch-bot.com/v1.0/scenes -H "Authorization: <api key>" -H "Content-type: application/json; charset=utf8"  -A "switchbot"Results:

Code: [Select]

{"statusCode":100,"body":[{"sceneId":"T01-202011091019-34589658","sceneName":"XBox On"},{"sceneId":"T01-202011091101-35496984","sceneName":"XBox Off"},{"sceneId":"T01-202102022309-99256608","sceneName":"Test Scene"}],"message":"success"}
I disabled firewall and anything that could cause it to not work and still not working. I'm I missing something here or do I need to update some dll to support this scrape?

Thank you all in advance.

[jitterjames]

Is it just this webpage / endpoint or do you get the same error no matter which part of the api you try to access?

[jitterjames]

Also, I recommend you try with VC version 2.3.1.1

[jitterjames]

The following macro works fine for me in VC 2.3.1.1 except that I have no scenes because I don't have any hardware.

Code: [Select]

<?xml version="1.0" encoding="utf-16"?>
<!--VoxCommando 2.3.1.1-->
<command id="1273" name="Test get scenes" enabled="true" alwaysOn="False" confirm="False" requiredConfidence="0" loop="False" loopDelay="0" loopMax="0" description="">
  <action>
    <cmdType>Scrape.Get</cmdType>
    <params>
      <param>https://api.switch-bot.com/v1.0/scenes</param>
      <param />
      <param />
      <param />
      <param />
      <param>Authorization: b345lkj6h345klj6h345l6kj534h6354lkj6h354lkj654234lk5j3452</param>
    </params>
    <cmdRepeat>1</cmdRepeat>
  </action>
</command>

[Powers]

Updating to the beta version 2.3.1.1 worked! Do you think a dll file was corrupted or was something changed/added to scrape action?

Thank you James!

[jitterjames]

You're welcome. 

I made changes to the scrape actions based on your previous post from a while ago.

I needed to add some different security protocol options because there are so many different standards in use by different servers.

[Powers]

Its been a while but this is a follow up and a good step moving forward if you do have Switchbot devices. There are two generating commands, one for pulling devices and another for scenes and put to payload file.

To use:
1. Replace "[long api key]" (including brackets) with your own API key.
2. Replace "[scene ID "T01-*"]" with the scenes ID, usually starts with T01.

[jitterjames]

Thanks Powers!