Issues with Windows CLI "Logging Create" function

General
1

Is it at all possible to use the “logging … create” function to create a custom logging endpoint using the Windows CLI? If so, I haven’t been able to figure it out and would like to know how to do it. (We’re migrating from Edgio and are trying to have everything updated before our Edgio contract expires at the end of December.)

In Windows, the “format” argument needs to be escaped and nothing I’ve tried will validate once the CLI posts to the API endpoint. I’ve tried using Sumo Logic’s payload example as well as Fastly’s.
[Examples are in Github link below since “new forum users” can only post 2 links. #SMH]

I can create logging for either of the above using the web UI, but the CLI is not working and I’m not sure if it’s not “working for me”, “working on Windows” or “working at all” at this point.

I’ve posted more information on Github recommend that the “format” argument supports a file path as well as a string (like the VCL Create function).

1 Like
2

Hello again! Thanks for coming over here :slight_smile:

One thing that would be useful is running the command with the environment variable FASTLY_DEBUG_MODE set to true. This will cause the CLI to show you the actual API request and response involved (with the token redacted). With that in place you can see how the format was ‘modified’ on the way into the CLI. I’ll also increase your user level in the forum so you aren’t so limited in posting links/etc.

3

Actually… hold off on that. I can make a new version of the CLI later today which supports a ‘file path’ for the logging format parameter. It won’t be an official release but it will be something you can use to get you moving. Stay tuned.

4

A snapshot build with this new feature was sent to @JamoCA ; we’ll get this added for all logging commands soon and published as a proper release.

5

ok. Thanks!

Being able to define a file path will be a lot more beneficial as it will remove a conversion/rewrite step by separating the BAT file logic from the “format” payload.

The FASTLY_DEBUG_MODE tip helped me a lot. I was able to contrast & compare to figure out what the DOS command line was doing and how it was interpreted by the CLI & API. I wrote a script to convert the “format” argument string to make it Windows “command line-friendly” and got it finally work.

2 Likes
6

We will probably do an audit of all the CLI commands to find other string parameters that are likely to not be ‘simple strings’ and make this change for all of them.

7

I don’t think the new format w/“path” attribute is working. There’s no error in the CLI, but the web UI indicates that it “Failed to parse log format string” that was added via the CLI API.

I’ve attempted to copy-and-paste the log format string from Log streaming: Sumo Logic . (I tried clicking the “copy” button, but it doesn’t work. A toast modal indicates “copied to clipboard”, but nothing happens. I’m using MS EdgeDev and have added similar clipboard features to my web applications, so I’m not sure why it’s not working for me from the Fastly website.)

Here’s the command line:

fastly.exe logging sumologic create --service-id=%SITEID% --name="%LOGNAME%" --url="%COLLECTORURL%" --version latest --message-type=blank --autoclone --format=./sumologic-test.txt

and it creates a log entry that appears to be correct, but it can’t be parsed. (???)

Here’s the “sumologic-test.txt” file that I’m using in my test. It matches the content from the Fastly documentation for Sumo Logic.

{
  "timestamp": "%{strftime(\{"%Y-%m-%dT%H:%M:%S%z"\}, time.start)}V",
  "client_ip": "%{req.http.Fastly-Client-IP}V",
  "geo_country": "%{client.geo.country_name}V",
  "geo_city": "%{client.geo.city}V",
  "host": "%{if(req.http.Fastly-Orig-Host, req.http.Fastly-Orig-Host, req.http.Host)}V",
  "url": "%{json.escape(req.url)}V",
  "request_method": "%{json.escape(req.method)}V",
  "request_protocol": "%{json.escape(req.proto)}V",
  "request_referer": "%{json.escape(req.http.referer)}V",
  "request_user_agent": "%{json.escape(req.http.User-Agent)}V",
  "response_state": "%{json.escape(fastly_info.state)}V",
  "response_status": %{resp.status}V,
  "response_reason": %{if(resp.response, "%22"+json.escape(resp.response)+"%22", "null")}V,
  "response_body_size": %{resp.body_bytes_written}V,
  "fastly_server": "%{json.escape(server.identity)}V",
  "fastly_is_edge": %{if(fastly.ff.visits_this_service == 0, "true", "false")}V
}

Is it possible that the example code for Sumo Logic is untested and not valid to begin with?

Please advise. Thanks.

8

It’s certainly possible, I’ll try to test this today. Are you able to create a logging endpoint using that same format string using the Fastly Control Panel?

9

Ahh, unfortunately I cannot test this as I don’t have a Sumo Logic account to connect to. At this point it would be best if this was handled through a Zendesk ticket, as we’ll need to bring in some assistance from our logging team. Do you have a ticket already created?

10

FYI: I just retested it by clearing out the CDN’s logging configuration and creating a new one using CLI with the format=filepath syntax, and it worked.

NOTE: I tested this by copying the working configuration from the site, saving it as a TXT file and then passing the file path (format=sumologic.txt). There may have been something wrong with the original string that I shared; especially since I was experiencing issues regarding which characters were being successfully escaped on the Windows command line. #notFun

Thanks for adding this feature. Hopefully, any other requests that require potentially unfriendly OS characters can be treated in this manner. (NOTE: I recently tried to use git clone on a wiki, only to find that the author used illegal colons : in the filenames. #smh)

11

Great news, thanks! We’ll be releasing a new version of the CLI ‘soon’ which will extend this --format=<path> support to all the subcommands where it makes sense.

1 Like
12

Actually, I take it back. I tried creating a new logger configuration for New Relic and the event messages were being logged as “./newrelic.txt” rather than the content of the newrelic.txt file. I tried some variations and the following syntax didn’t work. I ran this command from within the same sub-directory that contained the EXE, BAT and TXT files.

--format=newrelic-server1.txt
--format="newrelic-server1.txt"
--format="./newrelic-server1.txt"

--format=newrelic-%server%.txt
--format=./newrelic-%server%.txt
--format="./newrelic-%server%.txt"

I’m not sure if the CLI validates the contents of the file and then found it to be invalid and instead chose to use the text string instead.

NOTE: I was able to get the format string to work inline by passing the entire copied FORMAT example string from the Fastly website through an “escape-for-CLI” function that I wrote so that it wouldn’t throw an error when used in a Windows BAT file.

13

Yes, unfortunately the code that handles --format= isn’t great (and this is on the list to improve). If it can’t open the file at the stated path, or can’t read it after opening it, then the string provided is sent directly to the API without any warning or error message. It does not validate the contents of the file as it cannot do that :slight_smile:

14

My apologies, I should have noticed this earlier. The special build of the CLI that I gave you only supports file paths for the --format argument to logging create sumologic, not all logging create ... commands. We’ll get that addressed in a CLI release in the near future.

1 Like