How to Output Java Logs in JSON

🕒Reading Time: 5 mins

As applications grow, so does the need for structured logs that aid in debugging, fuel analytics, and provide real-time insights. JSON-based logging is perfect for those who want structured, readable logs. Here’s a quick guide to setting up JSON logging in Java with Logback, Log4j2, SLF4J, and Java Util Logging (JUL).

Why JSON Logging?

JSON logs are structured, flexible, and highly readable by log management tools, offering:

Structured Data: Capture complex data as JSON objects.
Enhanced Searchability: JSON-friendly tools like Kibana and Splunk simplify large-scale log management.
Integration: JSON logs are widely compatible with distributed systems.
Data Insights: Easier analysis of performance, errors, and other metrics.

Logback Configuration (Also for Spring Boot)

To set up JSON logging with Logback (including Spring Boot applications):

Dependencies in pom.xml:

   <dependency>
       <groupId>ch.qos.logback</groupId>
       <artifactId>logback-classic</artifactId>
       <version>1.2.3</version>
   </dependency>
   <dependency>
       <groupId>net.logstash.logback</groupId>
       <artifactId>logstash-logback-encoder</artifactId>
       <version>6.6</version>
   </dependency>

Configuration in logback.xml:

   <configuration>
       <appender name="JSON_CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
           <encoder class="net.logstash.logback.encoder.LogstashEncoder">
               <customFields>{"app_name":"MyApp","env":"production"}</customFields>
           </encoder>
       </appender>
       <root level="info">
           <appender-ref ref="JSON_CONSOLE" />
       </root>
   </configuration>

Java Code Example:

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class Application {
    private static final Logger logger = LoggerFactory.getLogger(Application.class);
    public static void main(String[] args) {
        logger.info("Application started");
    }
}

Sample JSON Output:

{
    "@timestamp": "2024-11-03T10:15:30.000Z",
    "level": "INFO",
    "logger_name": "com.example.Application",
    "message": "Application started",
    "app_name": "MyApp",
    "env": "production"
}

Log4j2 Configuration

Log4j2 supports JSON output directly. This example applies specifically to Log4j version 2.

Dependency:

   <dependency>
       <groupId>org.apache.logging.log4j</groupId>
       <artifactId>log4j-core</artifactId>
       <version>2.14.1</version>
   </dependency>

Configuration in log4j2.xml:

   <Configuration status="WARN">
       <Appenders>
           <Console name="Console" target="SYSTEM_OUT">
               <JsonLayout compact="true" eventEol="true">
                   <KeyValuePair key="app_name" value="MyApp"/>
                   <KeyValuePair key="env" value="production"/>
               </JsonLayout>
           </Console>
       </Appenders>
       <Loggers>
           <Root level="info">
               <AppenderRef ref="Console"/>
           </Root>
       </Loggers>
   </Configuration>

Java Code Example:

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

public class Application {
    private static final Logger logger = LogManager.getLogger(Application.class);
    public static void main(String[] args) {
        logger.info("Application started");
    }
}

Sample JSON Output:

{
    "timeMillis": 1609459200000,
    "thread": "main",
    "level": "INFO",
    "loggerName": "com.example.Application",
    "message": "Application started",
    "app_name": "MyApp",
    "env": "production"
}

Log4j 1.x with Jackson and ObjectMapper

For Log4j version 1.x, you can format JSON logs using Jackson’s ObjectMapper.

Dependency for Jackson:

   <dependency>
       <groupId>com.fasterxml.jackson.core</groupId>
       <artifactId>jackson-databind</artifactId>
       <version>2.12.3</version>
   </dependency>

Log JSON with ObjectMapper:

   import com.fasterxml.jackson.databind.ObjectMapper;
   import org.apache.log4j.Logger;

   public class Application {
       private static final Logger logger = Logger.getLogger(Application.class);
       private static final ObjectMapper mapper = new ObjectMapper();

       public static void main(String[] args) {
           try {
               String jsonLog = mapper.writeValueAsString(new LogEvent("INFO", 
                    "Application started", "MyApp", "production"));
               logger.info(jsonLog);
           } catch (Exception e) {
               logger.error("Error logging JSON", e);
           }
       }
   }

   class LogEvent {
       public String level;
       public String message;
       public String appName;
       public String env;

       public LogEvent(String level, String message, String appName, String env) {
           this.level = level;
           this.message = message;
           this.appName = appName;
           this.env = env;
       }
   }

Sample JSON Output:

{
    "level": "INFO",
    "message": "Application started",
    "appName": "MyApp",
    "env": "production"
}

Java Util Logging (JUL) with Custom JSON Formatter

For JUL, use a custom formatter to output JSON.

Create the JsonFormatter Class:

   import java.util.logging.*;
   import org.json.JSONObject;

   public class JsonFormatter extends Formatter {
       @Override
       public String format(LogRecord record) {
           JSONObject json = new JSONObject();
           json.put("level", record.getLevel().getName());
           json.put("logger", record.getLoggerName());
           json.put("message", record.getMessage());
           json.put("timestamp", record.getMillis());
           json.put("app_name", "MyApp");
           json.put("env", "production");
           return json.toString(4) + "\n";
       }
   }

Apply the Formatter:

   Logger logger = Logger.getLogger("MyLogger");
   ConsoleHandler consoleHandler = new ConsoleHandler();
   consoleHandler.setFormatter(new JsonFormatter());
   logger.addHandler(consoleHandler);

Java Code Example:

public class Application {
    private static final Logger logger = Logger.getLogger(Application.class.getName());
    public static void main(String[] args) {
        logger.info("Application started");
    }
}

Sample JSON Output:

{
    "level": "INFO",
    "logger": "MyLogger",
    "message": "Application started",
    "timestamp": 1609459200000,
    "app_name": "MyApp",
    "env": "production"
}

Best Practices for JSON Logging

Appropriate Log Levels: Avoid unnecessary data in production logs.
Custom Fields: Include essential metadata without clutter.
Secure Sensitive Data: Avoid logging sensitive information.

Wrapping Up

JSON logging in Java offers clarity and structure, making logs easier to parse and analyze. Try JSON logging today for powerful insights and streamlined debugging.