C#•4mo ago

ASP.NET endpoint logging

I am building an ASP.NET api. I want to log info about who is accessing my API and how. I want to log the IP address of the user, the end point they called and the parameters they passed ind and the current time. I've done some research on this already and found Serilog, adding it to my middleware

PathString endpoint = ctx.Request.Path;
string? ipAddress = ctx.Connection.RemoteIpAddress?.ToString();
string? currentTime = DateTimeOffset.Now.ToString("yyyy-MM-dd HH:mm:ss.fff zzz");
string requestData = await GetRequestBodyAsync(ctx.Request);

// Log to the endpoint-specific file
Log.Logger.ForContext("SourceContext", "API_Endpoints")
    .Information("{Time}: IP: {IP}, Endpoint: {Endpoint}, Data: {Data}",
        currentTime, ipAddress, endpoint, requestData);

PathString endpoint = ctx.Request.Path;
string? ipAddress = ctx.Connection.RemoteIpAddress?.ToString();
string? currentTime = DateTimeOffset.Now.ToString("yyyy-MM-dd HH:mm:ss.fff zzz");
string requestData = await GetRequestBodyAsync(ctx.Request);

// Log to the endpoint-specific file
Log.Logger.ForContext("SourceContext", "API_Endpoints")
    .Information("{Time}: IP: {IP}, Endpoint: {Endpoint}, Data: {Data}",
        currentTime, ipAddress, endpoint, requestData);

But this logs everything, I need to log less. For one thing, one of my endpoints is a login and i don't want to store passwords in the logs. For another this system logs stuff that isn't and API calls. here's a snippet of what it captures:

10 Replies

hutoanhillOP•4mo ago

2024-09-07 22:22:34.097 -05:00 [INF] Request finished HTTP/1.1 GET http://localhost:5154/swagger/index.html - 200 null text/html;charset=utf-8 233.4375ms
2024-09-07 22:22:34.136 -05:00 [INF] Request starting HTTP/1.1 GET http://localhost:5154/swagger/swagger-ui.css - null null
2024-09-07 22:22:34.140 -05:00 [INF] Message received: 'null'
2024-09-07 22:22:34.145 -05:00 [INF] Request starting HTTP/1.1 GET http://localhost:5154/swagger/index.js - null null
2024-09-07 22:22:34.150 -05:00 [INF] Message received: 'null'
2024-09-07 22:22:34.151 -05:00 [INF] Request finished HTTP/1.1 GET http://localhost:5154/swagger/index.js - 200 null application/javascript;charset=utf-8 7.0827ms
2024-09-07 22:22:34.168 -05:00 [INF] Sending file. Request path: '/swagger-ui.css'. Physical path: 'N/A'
2024-09-07 22:22:34.168 -05:00 [INF] Request finished HTTP/1.1 GET http://localhost:5154/swagger/swagger-ui.css - 499 152034 text/css 32.0393ms
2024-09-07 22:22:34.255 -05:00 [INF] Request starting HTTP/1.1 GET http://localhost:5154/swagger/favicon-16x16.png - null null

2024-09-07 22:22:34.097 -05:00 [INF] Request finished HTTP/1.1 GET http://localhost:5154/swagger/index.html - 200 null text/html;charset=utf-8 233.4375ms
2024-09-07 22:22:34.136 -05:00 [INF] Request starting HTTP/1.1 GET http://localhost:5154/swagger/swagger-ui.css - null null
2024-09-07 22:22:34.140 -05:00 [INF] Message received: 'null'
2024-09-07 22:22:34.145 -05:00 [INF] Request starting HTTP/1.1 GET http://localhost:5154/swagger/index.js - null null
2024-09-07 22:22:34.150 -05:00 [INF] Message received: 'null'
2024-09-07 22:22:34.151 -05:00 [INF] Request finished HTTP/1.1 GET http://localhost:5154/swagger/index.js - 200 null application/javascript;charset=utf-8 7.0827ms
2024-09-07 22:22:34.168 -05:00 [INF] Sending file. Request path: '/swagger-ui.css'. Physical path: 'N/A'
2024-09-07 22:22:34.168 -05:00 [INF] Request finished HTTP/1.1 GET http://localhost:5154/swagger/swagger-ui.css - 499 152034 text/css 32.0393ms
2024-09-07 22:22:34.255 -05:00 [INF] Request starting HTTP/1.1 GET http://localhost:5154/swagger/favicon-16x16.png - null null

In my ideal world, i could implement a system that tracks daily API calls and stores them in a list of the below datatype:

public class EndpointLog {
    public IPAddress Address { get; set; }
    
    public string Endpoint { get; set; }

    public JObject Request { get; set; }
    
    public DateTime Time { get; set; }
}

public class EndpointLog {
    public IPAddress Address { get; set; }
    
    public string Endpoint { get; set; }

    public JObject Request { get; set; }
    
    public DateTime Time { get; set; }
}

This would let me extract info about API calls and return it programaticly. I've tinkered with converting all my API request data types to share a common IRequest interface that would let me convert them to JSON without capturing sensitive info like passwords, but i cant figure out how to get that datatype out of the request body, or if that's even possible. This seems like a problem other people would have, so i assume there is a prebaked library out there to do this, or something very close either that or someone came up with a smarter system and i can learn about that. There is the option of adding code to every endpoint for capturing this, but that feels.. clunky... like i am totally going to forget to add this to some endpoint. There should be some way of capturing this data generaly for any endpoint

HtmlCompiler•4mo ago

it's not entirely clear to me what the model is for but i think what you are looking for is a middleware to add to httpclient and maybe hiding the raw logs with serilog settings

hutoanhillOP•4mo ago

The code ive got above is adding a middlewhare using WebApplication.Use() and what do you mean by the model?

Angius•4mo ago

If you want some data to not be logged, you can decorate it with an attribute and filter it out in the middleware

hutoanhillOP•4mo ago

dont know how to do this. ive only ever used attributes as part of existing systems like ASP.NET or Discord.NET. i dont know how to make my own or do anything with them. can you point me to the place to go to learn more about them?

Angius•4mo ago

Attributes are just metadata. You can use them either with reflections or source generators

Angius•4mo ago

There's also this Serilog enricher: https://github.com/serilog-contrib/Serilog.Enrichers.Sensitive

GitHub

GitHub - serilog-contrib/Serilog.Enrichers.Sensitive: A Serilog Log...

A Serilog LogEvent enricher that masks sensitive data - serilog-contrib/Serilog.Enrichers.Sensitive

hutoanhillOP•4mo ago

ok. i went and found a crash corse on attributes not sure how this is helpful... heres my code for getting the request data (just realized i didnt include that sry)

private static async Task<string> GetRequestBodyAsync(HttpRequest request){
    request.EnableBuffering();
    StreamReader bodyStream = new StreamReader(request.Body);
    string body = await bodyStream.ReadToEndAsync();
    request.Body.Position = 0;
    return body;
}

private static async Task<string> GetRequestBodyAsync(HttpRequest request){
    request.EnableBuffering();
    StreamReader bodyStream = new StreamReader(request.Body);
    string body = await bodyStream.ReadToEndAsync();
    request.Body.Position = 0;
    return body;
}

This doesn't parse the request into a class, it just returns a json object as a string. is there a way to figure out which type of request its using and parse it into that type? ok, i might have a solution. I've created custom versions of the request datatypes adding decorators to sensive info, then i added this middleware:

hutoanhillOP•4mo ago

message.txt

hutoanhillOP•4mo ago

lets hope this works... well, its not working... but that might be the rest of the API nope its not working is theis this the right way to impliment this middleware? app.UseMiddleware<ApiLoggingMiddleware>();

Gaming

Programming

ASP.NET endpoint logging