Rewrite logger system (#24726)

## ⚠️ Breaking

The `log.<mode>.<logger>` style config has been dropped. If you used it,
please check the new config manual & app.example.ini to make your
instance output logs as expected.

Although many legacy options still work, it's encouraged to upgrade to
the new options.

The SMTP logger is deleted because SMTP is not suitable to collect logs.

If you have manually configured Gitea log options, please confirm the
logger system works as expected after upgrading.

## Description

Close #12082 and maybe more log-related issues, resolve some related
FIXMEs in old code (which seems unfixable before)

Just like rewriting queue #24505 : make code maintainable, clear legacy
bugs, and add the ability to support more writers (eg: JSON, structured
log)

There is a new document (with examples): `logging-config.en-us.md`

This PR is safer than the queue rewriting, because it's just for
logging, it won't break other logic.

## The old problems

The logging system is quite old and difficult to maintain:
* Unclear concepts: Logger, NamedLogger, MultiChannelledLogger,
SubLogger, EventLogger, WriterLogger etc
* Some code is diffuclt to konw whether it is right:
`log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs
`log.DelLogger("console")`
* The old system heavily depends on ini config system, it's difficult to
create new logger for different purpose, and it's very fragile.
* The "color" trick is difficult to use and read, many colors are
unnecessary, and in the future structured log could help
* It's difficult to add other log formats, eg: JSON format
* The log outputer doesn't have full control of its goroutine, it's
difficult to make outputer have advanced behaviors
* The logs could be lost in some cases: eg: no Fatal error when using
CLI.
* Config options are passed by JSON, which is quite fragile.
* INI package makes the KEY in `[log]` section visible in `[log.sub1]`
and `[log.sub1.subA]`, this behavior is quite fragile and would cause
more unclear problems, and there is no strong requirement to support
`log.<mode>.<logger>` syntax.


## The new design

See `logger.go` for documents.


## Screenshot

<details>


![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff)


![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9)


![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee)

</details>

## TODO

* [x] add some new tests
* [x] fix some tests
* [x] test some sub-commands (manually ....)

---------

Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Giteabot <teabot@gitea.io>
pull/24827/head^2
wxiaoguang 2023-05-22 06:35:11 +08:00 committed by GitHub
parent 65dff8e364
commit 4647660776
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
109 changed files with 3806 additions and 5337 deletions

View File

@ -9,6 +9,7 @@ import (
"context"
"errors"
"fmt"
"io"
"os"
"os/signal"
"strings"
@ -59,7 +60,7 @@ func confirm() (bool, error) {
func initDB(ctx context.Context) error {
setting.Init(&setting.Options{})
setting.LoadDBSetting()
setting.InitSQLLog(false)
setting.InitSQLLoggersForCli(log.INFO)
if setting.Database.Type == "" {
log.Fatal(`Database settings are missing from the configuration file: %q.
@ -93,3 +94,17 @@ func installSignals() (context.Context, context.CancelFunc) {
return ctx, cancel
}
func setupConsoleLogger(level log.Level, colorize bool, out io.Writer) {
if out != os.Stdout && out != os.Stderr {
panic("setupConsoleLogger can only be used with os.Stdout or os.Stderr")
}
writeMode := log.WriterMode{
Level: level,
Colorize: colorize,
WriterOption: log.WriterConsoleOption{Stderr: out == os.Stderr},
}
writer := log.NewEventWriterConsole("console-default", writeMode)
log.GetManager().GetLogger(log.DEFAULT).RemoveAllWriters().AddWriters(writer)
}

View File

@ -4,10 +4,10 @@
package cmd
import (
"errors"
"fmt"
golog "log"
"os"
"path/filepath"
"strings"
"text/tabwriter"
@ -82,23 +82,25 @@ You should back-up your database before doing this and ensure that your database
}
func runRecreateTable(ctx *cli.Context) error {
// Redirect the default golog to here
golog.SetFlags(0)
golog.SetPrefix("")
golog.SetOutput(log.NewLoggerAsWriter("INFO", log.GetLogger(log.DEFAULT)))
setting.Init(&setting.Options{})
setting.LoadDBSetting()
setting.Log.EnableXORMLog = ctx.Bool("debug")
setting.Database.LogSQL = ctx.Bool("debug")
// FIXME: don't use CfgProvider directly
setting.CfgProvider.Section("log").Key("XORM").SetValue(",")
setting.InitSQLLog(!ctx.Bool("debug"))
stdCtx, cancel := installSignals()
defer cancel()
// Redirect the default golog to here
golog.SetFlags(0)
golog.SetPrefix("")
golog.SetOutput(log.LoggerToWriter(log.GetLogger(log.DEFAULT).Info))
debug := ctx.Bool("debug")
setting.Init(&setting.Options{})
setting.LoadDBSetting()
if debug {
setting.InitSQLLoggersForCli(log.DEBUG)
} else {
setting.InitSQLLoggersForCli(log.INFO)
}
setting.Database.LogSQL = debug
if err := db.InitEngine(stdCtx); err != nil {
fmt.Println(err)
fmt.Println("Check if you are using the right config file. You can use a --config directive to specify one.")
@ -125,44 +127,31 @@ func runRecreateTable(ctx *cli.Context) error {
})
}
func setDoctorLogger(ctx *cli.Context) {
func setupDoctorDefaultLogger(ctx *cli.Context, colorize bool) {
// Silence the default loggers
setupConsoleLogger(log.FATAL, log.CanColorStderr, os.Stderr)
logFile := ctx.String("log-file")
if !ctx.IsSet("log-file") {
logFile = "doctor.log"
}
colorize := log.CanColorStdout
if ctx.IsSet("color") {
colorize = ctx.Bool("color")
}
if len(logFile) == 0 {
log.NewLogger(1000, "doctor", "console", fmt.Sprintf(`{"level":"NONE","stacktracelevel":"NONE","colorize":%t}`, colorize))
// if no doctor log-file is set, do not show any log from default logger
return
}
defer func() {
recovered := recover()
if recovered == nil {
if logFile == "-" {
setupConsoleLogger(log.TRACE, colorize, os.Stdout)
} else {
logFile, _ = filepath.Abs(logFile)
writeMode := log.WriterMode{Level: log.TRACE, WriterOption: log.WriterFileOption{FileName: logFile}}
writer, err := log.NewEventWriter("console-to-file", "file", writeMode)
if err != nil {
log.FallbackErrorf("unable to create file log writer: %v", err)
return
}
err, ok := recovered.(error)
if !ok {
panic(recovered)
}
if errors.Is(err, os.ErrPermission) {
fmt.Fprintf(os.Stderr, "ERROR: Unable to write logs to provided file due to permissions error: %s\n %v\n", logFile, err)
} else {
fmt.Fprintf(os.Stderr, "ERROR: Unable to write logs to provided file: %s\n %v\n", logFile, err)
}
fmt.Fprintf(os.Stderr, "WARN: Logging will be disabled\n Use `--log-file` to configure log file location\n")
log.NewLogger(1000, "doctor", "console", fmt.Sprintf(`{"level":"NONE","stacktracelevel":"NONE","colorize":%t}`, colorize))
}()
if logFile == "-" {
log.NewLogger(1000, "doctor", "console", fmt.Sprintf(`{"level":"trace","stacktracelevel":"NONE","colorize":%t}`, colorize))
} else {
log.NewLogger(1000, "doctor", "file", fmt.Sprintf(`{"filename":%q,"level":"trace","stacktracelevel":"NONE"}`, logFile))
log.GetManager().GetLogger(log.DEFAULT).RemoveAllWriters().AddWriters(writer)
}
}
@ -170,22 +159,17 @@ func runDoctor(ctx *cli.Context) error {
stdCtx, cancel := installSignals()
defer cancel()
// Silence the default loggers
log.DelNamedLogger("console")
log.DelNamedLogger(log.DEFAULT)
// Now setup our own
setDoctorLogger(ctx)
colorize := log.CanColorStdout
if ctx.IsSet("color") {
colorize = ctx.Bool("color")
}
// Finally redirect the default golog to here
setupDoctorDefaultLogger(ctx, colorize)
// Finally redirect the default golang's log to here
golog.SetFlags(0)
golog.SetPrefix("")
golog.SetOutput(log.NewLoggerAsWriter("INFO", log.GetLogger(log.DEFAULT)))
golog.SetOutput(log.LoggerToWriter(log.GetLogger(log.DEFAULT).Info))
if ctx.IsSet("list") {
w := tabwriter.NewWriter(os.Stdout, 0, 8, 1, '\t', 0)
@ -233,17 +217,5 @@ func runDoctor(ctx *cli.Context) error {
}
}
// Now we can set up our own logger to return information about what the doctor is doing
if err := log.NewNamedLogger("doctorouter",
0,
"console",
"console",
fmt.Sprintf(`{"level":"INFO","stacktracelevel":"NONE","colorize":%t,"flags":-1}`, colorize)); err != nil {
fmt.Println(err)
return err
}
logger := log.GetLogger("doctorouter")
defer logger.Close()
return doctor.RunChecks(stdCtx, logger, ctx.Bool("fix"), checks)
return doctor.RunChecks(stdCtx, colorize, ctx.Bool("fix"), checks)
}

View File

@ -172,10 +172,7 @@ func runDump(ctx *cli.Context) error {
outType := ctx.String("type")
if fileName == "-" {
file = os.Stdout
err := log.DelLogger("console")
if err != nil {
fatal("Deleting default logger failed. Can not write to stdout: %v", err)
}
setupConsoleLogger(log.FATAL, log.CanColorStderr, os.Stderr)
} else {
for _, suffix := range outputTypeEnum.Enum {
if strings.HasSuffix(fileName, "."+suffix) {

View File

@ -97,13 +97,7 @@ type assetFile struct {
}
func initEmbeddedExtractor(c *cli.Context) error {
// FIXME: there is a bug, if the user runs `gitea embedded` with a different user or root,
// The setting.Init (loadRunModeFrom) will fail and do log.Fatal
// But the console logger has been deleted, so nothing is printed, the user sees nothing and Gitea just exits.
// Silence the console logger
log.DelNamedLogger("console")
log.DelNamedLogger(log.DEFAULT)
setupConsoleLogger(log.ERROR, log.CanColorStderr, os.Stderr)
// Read configuration file
setting.Init(&setting.Options{

View File

@ -16,13 +16,13 @@ import (
var (
defaultLoggingFlags = []cli.Flag{
cli.StringFlag{
Name: "group, g",
Usage: "Group to add logger to - will default to \"default\"",
Name: "logger",
Usage: `Logger name - will default to "default"`,
}, cli.StringFlag{
Name: "name, n",
Usage: "Name of the new logger - will default to mode",
Name: "writer",
Usage: "Name of the log writer - will default to mode",
}, cli.StringFlag{
Name: "level, l",
Name: "level",
Usage: "Logging level for the new logger",
}, cli.StringFlag{
Name: "stacktrace-level, L",
@ -83,8 +83,8 @@ var (
cli.BoolFlag{
Name: "debug",
}, cli.StringFlag{
Name: "group, g",
Usage: "Group to add logger to - will default to \"default\"",
Name: "logger",
Usage: `Logger name - will default to "default"`,
},
},
Action: runRemoveLogger,
@ -93,15 +93,6 @@ var (
Usage: "Add a logger",
Subcommands: []cli.Command{
{
Name: "console",
Usage: "Add a console logger",
Flags: append(defaultLoggingFlags,
cli.BoolFlag{
Name: "stderr",
Usage: "Output console logs to stderr - only relevant for console",
}),
Action: runAddConsoleLogger,
}, {
Name: "file",
Usage: "Add a file logger",
Flags: append(defaultLoggingFlags, []cli.Flag{
@ -148,28 +139,6 @@ var (
},
}...),
Action: runAddConnLogger,
}, {
Name: "smtp",
Usage: "Add an SMTP logger",
Flags: append(defaultLoggingFlags, []cli.Flag{
cli.StringFlag{
Name: "username, u",
Usage: "Mail server username",
}, cli.StringFlag{
Name: "password, P",
Usage: "Mail server password",
}, cli.StringFlag{
Name: "host, H",
Usage: "Mail server host (defaults to: 127.0.0.1:25)",
}, cli.StringSliceFlag{
Name: "send-to, s",
Usage: "Email address(es) to send to",
}, cli.StringFlag{
Name: "subject, S",
Usage: "Subject header of sent emails",
},
}...),
Action: runAddSMTPLogger,
},
},
}, {
@ -194,50 +163,16 @@ func runRemoveLogger(c *cli.Context) error {
defer cancel()
setup(ctx, c.Bool("debug"))
group := c.String("group")
if len(group) == 0 {
group = log.DEFAULT
logger := c.String("logger")
if len(logger) == 0 {
logger = log.DEFAULT
}
name := c.Args().First()
writer := c.Args().First()
extra := private.RemoveLogger(ctx, group, name)
extra := private.RemoveLogger(ctx, logger, writer)
return handleCliResponseExtra(extra)
}
func runAddSMTPLogger(c *cli.Context) error {
ctx, cancel := installSignals()
defer cancel()
setup(ctx, c.Bool("debug"))
vals := map[string]interface{}{}
mode := "smtp"
if c.IsSet("host") {
vals["host"] = c.String("host")
} else {
vals["host"] = "127.0.0.1:25"
}
if c.IsSet("username") {
vals["username"] = c.String("username")
}
if c.IsSet("password") {
vals["password"] = c.String("password")
}
if !c.IsSet("send-to") {
return fmt.Errorf("Some recipients must be provided")
}
vals["sendTos"] = c.StringSlice("send-to")
if c.IsSet("subject") {
vals["subject"] = c.String("subject")
} else {
vals["subject"] = "Diagnostic message from Gitea"
}
return commonAddLogger(c, mode, vals)
}
func runAddConnLogger(c *cli.Context) error {
ctx, cancel := installSignals()
defer cancel()
@ -301,25 +236,12 @@ func runAddFileLogger(c *cli.Context) error {
return commonAddLogger(c, mode, vals)
}
func runAddConsoleLogger(c *cli.Context) error {
ctx, cancel := installSignals()
defer cancel()
setup(ctx, c.Bool("debug"))
vals := map[string]interface{}{}
mode := "console"
if c.IsSet("stderr") && c.Bool("stderr") {
vals["stderr"] = c.Bool("stderr")
}
return commonAddLogger(c, mode, vals)
}
func commonAddLogger(c *cli.Context, mode string, vals map[string]interface{}) error {
if len(c.String("level")) > 0 {
vals["level"] = log.FromString(c.String("level")).String()
vals["level"] = log.LevelFromString(c.String("level")).String()
}
if len(c.String("stacktrace-level")) > 0 {
vals["stacktraceLevel"] = log.FromString(c.String("stacktrace-level")).String()
vals["stacktraceLevel"] = log.LevelFromString(c.String("stacktrace-level")).String()
}
if len(c.String("expression")) > 0 {
vals["expression"] = c.String("expression")
@ -333,18 +255,18 @@ func commonAddLogger(c *cli.Context, mode string, vals map[string]interface{}) e
if c.IsSet("color") {
vals["colorize"] = c.Bool("color")
}
group := "default"
if c.IsSet("group") {
group = c.String("group")
logger := log.DEFAULT
if c.IsSet("logger") {
logger = c.String("logger")
}
name := mode
if c.IsSet("name") {
name = c.String("name")
writer := mode
if c.IsSet("writer") {
writer = c.String("writer")
}
ctx, cancel := installSignals()
defer cancel()
extra := private.AddLogger(ctx, group, name, mode, vals)
extra := private.AddLogger(ctx, logger, writer, mode, vals)
return handleCliResponseExtra(extra)
}

View File

@ -56,11 +56,10 @@ var CmdServ = cli.Command{
}
func setup(ctx context.Context, debug bool) {
_ = log.DelLogger("console")
if debug {
_ = log.NewLogger(1000, "console", "console", `{"level":"trace","stacktracelevel":"NONE","stderr":true}`)
setupConsoleLogger(log.TRACE, false, os.Stderr)
} else {
_ = log.NewLogger(1000, "console", "console", `{"level":"fatal","stacktracelevel":"NONE","stderr":true}`)
setupConsoleLogger(log.FATAL, false, os.Stderr)
}
setting.Init(&setting.Options{})
if debug {

View File

@ -103,11 +103,9 @@ func createPIDFile(pidPath string) {
func runWeb(ctx *cli.Context) error {
if ctx.Bool("verbose") {
_ = log.DelLogger("console")
log.NewLogger(0, "console", "console", fmt.Sprintf(`{"level": "trace", "colorize": %t, "stacktraceLevel": "none"}`, log.CanColorStdout))
setupConsoleLogger(log.TRACE, log.CanColorStdout, os.Stdout)
} else if ctx.Bool("quiet") {
_ = log.DelLogger("console")
log.NewLogger(0, "console", "console", fmt.Sprintf(`{"level": "fatal", "colorize": %t, "stacktraceLevel": "none"}`, log.CanColorStdout))
setupConsoleLogger(log.FATAL, log.CanColorStdout, os.Stdout)
}
defer func() {
if panicked := recover(); panicked != nil {
@ -156,7 +154,7 @@ func runWeb(ctx *cli.Context) error {
case <-graceful.GetManager().IsShutdown():
<-graceful.GetManager().Done()
log.Info("PID: %d Gitea Web Finished", os.Getpid())
log.Close()
log.GetManager().Close()
return err
default:
}
@ -199,7 +197,7 @@ func runWeb(ctx *cli.Context) error {
err := listen(c, true)
<-graceful.GetManager().Done()
log.Info("PID: %d Gitea Web Finished", os.Getpid())
log.Close()
log.GetManager().Close()
return err
}

View File

@ -230,7 +230,6 @@ RUN_MODE = ; prod
;;
;; Disable CDN even in "prod" mode
;OFFLINE_MODE = false
;DISABLE_ROUTER_LOG = false
;;
;; TLS Settings: Either ACME or manual
;; (Other common TLS configuration are found before)
@ -387,7 +386,7 @@ USER = root
;ITERATE_BUFFER_SIZE = 50
;;
;; Show the database generated SQL
LOG_SQL = false ; if unset defaults to true
;LOG_SQL = false
;;
;; Maximum number of DB Connect retries
;DB_RETRIES = 10
@ -550,34 +549,32 @@ ENABLE = true
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Main Logger
;;
;; Either "console", "file", "conn", "smtp" or "database", default is "console"
;; Either "console", "file" or "conn", default is "console"
;; Use comma to separate multiple modes, e.g. "console, file"
MODE = console
;;
;; Either "Trace", "Debug", "Info", "Warn", "Error", "Critical" or "None", default is "Info"
;; Either "Trace", "Debug", "Info", "Warn", "Error" or "None", default is "Info"
LEVEL = Info
;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Router Logger
;; Print Stacktrace with logs (rarely helpful, do not set) Either "Trace", "Debug", "Info", "Warn", "Error", default is "None"
;STACKTRACE_LEVEL = None
;;
;; Switch off the router log
;DISABLE_ROUTER_LOG=false
;; Buffer length of the channel, keep it as it is if you don't know what it is.
;BUFFER_LEN = 10000
;;
;; Set the log "modes" for the router log (if file is set the log file will default to router.log)
ROUTER = console
;; Sub logger modes, a single comma means use default MODE above, empty means disable it
;logger.access.MODE=
;logger.router.MODE=,
;logger.xorm.MODE=,
;;
;; The router will log different things at different levels.
;; Collect SSH logs (Creates log from ssh git request)
;;
;; * started messages will be logged at TRACE level
;; * polling/completed routers will be logged at INFO
;; * slow routers will be logged at WARN
;; * failed routers will be logged at WARN
;;
;; The routing level will default to that of the system but individual router level can be set in
;; [log.<mode>.router] LEVEL
;ENABLE_SSH_LOG = false
;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;; Access Logger (Creates log in NCSA common log format)
;;
;; Print request id which parsed from request headers in access log, when access log is enabled.
;; * E.g:
;; * In request Header: X-Request-ID: test-id-123
@ -587,57 +584,32 @@ ROUTER = console
;; If you configure more than one in the .ini file, it will match in the order of configuration,
;; and the first match will be finally printed in the log.
;; * E.g:
;; * In reuqest Header: X-Trace-ID: trace-id-1q2w3e4r
;; * In request Header: X-Trace-ID: trace-id-1q2w3e4r
;; * Configuration in app.ini: REQUEST_ID_HEADERS = X-Request-ID, X-Trace-ID, X-Req-ID
;; * Print in log: 127.0.0.1:58384 - - [14/Feb/2023:16:33:51 +0800] "trace-id-1q2w3e4r"
;;
;; REQUEST_ID_HEADERS =
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;; Access Logger (Creates log in NCSA common log format)
;;
;ENABLE_ACCESS_LOG = false
;;
;; Set the log "modes" for the access log (if file is set the log file will default to access.log)
;ACCESS = file
;REQUEST_ID_HEADERS =
;;
;; Sets the template used to create the access log.
;ACCESS_LOG_TEMPLATE = {{.Ctx.RemoteHost}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}" "{{.Ctx.Req.UserAgent}}"
;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;; SSH log (Creates log from ssh git request)
;;
;ENABLE_SSH_LOG = false
;;
;; Other Settings
;;
;; Print Stacktraces with logs. (Rarely helpful.) Either "Trace", "Debug", "Info", "Warn", "Error", "Critical", default is "None"
;STACKTRACE_LEVEL = None
;;
;; Buffer length of the channel, keep it as it is if you don't know what it is.
;BUFFER_LEN = 10000
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Creating specific log configuration
;;
;; You can set specific configuration for individual modes and subloggers
;; Log modes (aka log writers)
;;
;; Configuration available to all log modes/subloggers
;[log.%(WriterMode)]
;MODE=console/file/conn/...
;LEVEL=
;FLAGS = stdflags
;EXPRESSION =
;PREFIX =
;COLORIZE = false
;;
;; For "console" mode only
;[log.console]
;STDERR = false
;;
;; For "file" mode only
;LEVEL =
;; Set the file_name for the logger. If this is a relative path this
;; will be relative to ROOT_PATH
;[log.file]
;; Set the file_name for the logger. If this is a relative path this will be relative to ROOT_PATH
;FILE_NAME =
;; This enables automated log rotate(switch of following options), default is true
;LOG_ROTATE = true
@ -651,9 +623,8 @@ ROUTER = console
;COMPRESS = true
;; compression level see godoc for compress/gzip
;COMPRESSION_LEVEL = -1
;
;; For "conn" mode only
;LEVEL =
;;
;[log.conn]
;; Reconnect host for every single message, default is false
;RECONNECT_ON_MSG = false
;; Try to reconnect when connection is lost, default is false
@ -662,19 +633,6 @@ ROUTER = console
;PROTOCOL = tcp
;; Host address
;ADDR =
;
;; For "smtp" mode only
;LEVEL =
;; Name displayed in mail title, default is "Diagnostic message from server"
;SUBJECT = Diagnostic message from server
;; Mail server
;HOST =
;; Mailer user name and password
;USER =
;; Use PASSWD = `your password` for quoting if you use special characters in the password.
;PASSWD =
;; Receivers, can be one or more, e.g. 1@example.com,2@example.com
;RECEIVERS =
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

View File

@ -835,22 +835,16 @@ Default templates for project boards:
## Log (`log`)
- `ROOT_PATH`: **\<empty\>**: Root path for log files.
- `MODE`: **console**: Logging mode. For multiple modes, use a comma to separate values. You can configure each mode in per mode log subsections `\[log.modename\]`. By default the file mode will log to `$ROOT_PATH/gitea.log`.
- `MODE`: **console**: Logging mode. For multiple modes, use a comma to separate values. You can configure each mode in per mode log subsections `\[log.writer-mode-name\]`.
- `LEVEL`: **Info**: General log level. \[Trace, Debug, Info, Warn, Error, Critical, Fatal, None\]
- `STACKTRACE_LEVEL`: **None**: Default log level at which to log create stack traces. \[Trace, Debug, Info, Warn, Error, Critical, Fatal, None\]
- `STACKTRACE_LEVEL`: **None**: Default log level at which to log create stack traces (rarely useful, do not set it). \[Trace, Debug, Info, Warn, Error, Critical, Fatal, None\]
- `ENABLE_SSH_LOG`: **false**: save ssh log to log file
- `ENABLE_XORM_LOG`: **true**: Set whether to perform XORM logging. Please note SQL statement logging can be disabled by setting `LOG_SQL` to false in the `[database]` section.
### Router Log (`log`)
- `DISABLE_ROUTER_LOG`: **false**: Mute printing of the router log.
- `ROUTER`: **console**: The mode or name of the log the router should log to. (If you set this to `,` it will log to default Gitea logger.)
NB: You must have `DISABLE_ROUTER_LOG` set to `false` for this option to take effect. Configure each mode in per mode log subsections `\[log.modename.router\]`.
- `logger.access.MODE`: **\<empty\>**: The "access" logger
- `logger.router.MODE`: **,**: The "router" logger, a single comma means it will use the default MODE above
- `logger.xorm.MODE`: **,**: The "xorm" logger
### Access Log (`log`)
- `ENABLE_ACCESS_LOG`: **false**: Creates an access.log in NCSA common log format, or as per the following template
- `ACCESS`: **file**: Logging mode for the access logger, use a comma to separate values. Configure each mode in per mode log subsections `\[log.modename.access\]`. By default the file mode will log to `$ROOT_PATH/access.log`. (If you set this to `,` it will log to the default Gitea logger.)
- `ACCESS_LOG_TEMPLATE`: **`{{.Ctx.RemoteHost}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}" "{{.Ctx.Req.UserAgent}}"`**: Sets the template used to create the access log.
- The following variables are available:
- `Ctx`: the `context.Context` of the request.
@ -858,31 +852,31 @@ Default templates for project boards:
- `Start`: the start time of the request.
- `ResponseWriter`: the responseWriter from the request.
- `RequestID`: the value matching REQUEST_ID_HEADERSdefault: `-`, if not matched.
- You must be very careful to ensure that this template does not throw errors or panics as this template runs outside of the panic/recovery script.
- You must be very careful to ensure that this template does not throw errors or panics as this template runs outside the panic/recovery script.
- `REQUEST_ID_HEADERS`: **\<empty\>**: You can configure multiple values that are splited by comma here. It will match in the order of configuration, and the first match will be finally printed in the access log.
- e.g.
- In the Request Header: X-Request-ID: **test-id-123**
- Configuration in app.ini: REQUEST_ID_HEADERS = X-Request-ID
- Print in log: 127.0.0.1:58384 - - [14/Feb/2023:16:33:51 +0800] "**test-id-123**" ...
### Log subsections (`log.name`, `log.name.*`)
### Log subsections (`log.<writer-mode-name>`)
- `LEVEL`: **log.LEVEL**: Sets the log-level of this sublogger. Defaults to the `LEVEL` set in the global `[log]` section.
- `MODE`: **name**: Sets the mode of this log writer - Defaults to the provided subsection name. This allows you to have two different file loggers at different levels.
- `LEVEL`: **log.LEVEL**: Sets the log-level of this writer. Defaults to the `LEVEL` set in the global `[log]` section.
- `STACKTRACE_LEVEL`: **log.STACKTRACE_LEVEL**: Sets the log level at which to log stack traces.
- `MODE`: **name**: Sets the mode of this sublogger - Defaults to the provided subsection name. This allows you to have two different file loggers at different levels.
- `EXPRESSION`: **""**: A regular expression to match either the function name, file or message. Defaults to empty. Only log messages that match the expression will be saved in the logger.
- `FLAGS`: **stdflags**: A comma separated string representing the log flags. Defaults to `stdflags` which represents the prefix: `2009/01/23 01:23:23 ...a/b/c/d.go:23:runtime.Caller() [I]: message`. `none` means don't prefix log lines. See `modules/log/flags.go` for more information.
- `PREFIX`: **""**: An additional prefix for every log line in this logger. Defaults to empty.
- `COLORIZE`: **false**: Whether to colorize the log lines
### Console log mode (`log.console`, `log.console.*`, or `MODE=console`)
### Console log mode (`log.console`, or `MODE=console`)
- For the console logger `COLORIZE` will default to `true` if not on windows or the terminal is determined to be able to color.
- `STDERR`: **false**: Use Stderr instead of Stdout.
### File log mode (`log.file`, `log.file.*` or `MODE=file`)
### File log mode (`log.file`, or `MODE=file`)
- `FILE_NAME`: Set the file name for this logger. Defaults as described above. If relative will be relative to the `ROOT_PATH`
- `FILE_NAME`: Set the file name for this logger. Defaults to `gitea.log` (exception: access log defaults to `access.log`). If relative will be relative to the `ROOT_PATH`
- `LOG_ROTATE`: **true**: Rotate the log files.
- `MAX_SIZE_SHIFT`: **28**: Maximum size shift of a single file, 28 represents 256Mb.
- `DAILY_ROTATE`: **true**: Rotate logs daily.
@ -890,21 +884,13 @@ Default templates for project boards:
- `COMPRESS`: **true**: Compress old log files by default with gzip
- `COMPRESSION_LEVEL`: **-1**: Compression level
### Conn log mode (`log.conn`, `log.conn.*` or `MODE=conn`)
### Conn log mode (`log.conn`, or `MODE=conn`)
- `RECONNECT_ON_MSG`: **false**: Reconnect host for every single message.
- `RECONNECT`: **false**: Try to reconnect when connection is lost.
- `PROTOCOL`: **tcp**: Set the protocol, either "tcp", "unix" or "udp".
- `ADDR`: **:7020**: Sets the address to connect to.
### SMTP log mode (`log.smtp`, `log.smtp.*` or `MODE=smtp`)
- `USER`: User email address to send from.
- `PASSWD`: Password for the smtp server.
- `HOST`: **127.0.0.1:25**: The SMTP host to connect to.
- `RECEIVERS`: Email addresses to send to.
- `SUBJECT`: **Diagnostic message from Gitea**
## Cron (`cron`)
- `ENABLED`: **false**: Enable to run all cron tasks periodically with default settings.

View File

@ -0,0 +1,295 @@
---
date: "2019-04-02T17:06:00+01:00"
title: "Logging Configuration"
slug: "logging-config"
weight: 40
toc: false
draft: false
aliases:
- /en-us/logging-configuration
menu:
sidebar:
parent: "administration"
name: "Logging Configuration"
weight: 40
identifier: "logging-config"
---
# Logging Configuration
The logging configuration of Gitea mainly consists of 3 types of components:
- The `[log]` section for general configuration
- `[log.<mode-name>]` sections for the configuration of different log writers to output logs, aka: "writer mode", the mode name is also used as "writer name".
- The `[log]` section can also contain sub-logger configurations following the key schema `logger.<logger-name>.<CONFIG-KEY>`
There is a fully functional log output by default, so it is not necessary to define one.
**Table of Contents**
{{< toc >}}
## Collecting Logs for Help
To collect logs for help and issue report, see [Support Options]({{< relref "doc/help/support.en-us.md" >}}).
## The `[log]` section
Configuration of logging facilities in Gitea happen in the `[log]` section and its subsections.
In the top level `[log]` section the following configurations can be placed:
- `ROOT_PATH`: (Default: **%(GITEA_WORK_DIR)/log**): Base path for log files
- `MODE`: (Default: **console**) List of log outputs to use for the Default logger.
- `LEVEL`: (Default: **Info**) Least severe log events to persist, case-insensitive. Possible values are: `Trace`, `Debug`, `Info`, `Warn`, `Error`, `Fatal`.
- `STACKTRACE_LEVEL`: (Default: **None**) For this and more severe events the stacktrace will be printed upon getting logged.
And it can contain the following sub-loggers:
- `logger.router.MODE`: (Default: **,**): List of log outputs to use for the Router logger.
- `logger.access.MODE`: (Default: **\<empty\>**) List of log outputs to use for the Access logger. By default, the access logger is disabled.
- `logger.xorm.MODE`: (Default: **,**) List of log outputs to use for the XORM logger.
Setting a comma (`,`) to sub-logger's mode means making it use the default global `MODE`.
## Quick samples
### Default (empty) Configuration
The empty configuration is equivalent to default:
```ini
[log]
ROOT_PATH = %(GITEA_WORK_DIR)/log
MODE = console
LEVEL = Info
STACKTRACE_LEVEL = None
logger.router.MODE = ,
logger.xorm.MODE = ,
logger.access.MODE =
; this is the config options of "console" mode (used by MODE=console above)
[log.console]
MODE = console
FLAGS = stdflags
PREFIX =
COLORIZE = true
```
This is equivalent to sending all logs to the console, with default Golang log being sent to the console log too.
This is only a sample, and it is the default, do not need to write it into your configuration file.
### Disable Router logs and record some access logs to file
The Router logger is disabled, the access logs (>=Warn) goes into `access.log`:
```ini
[log]
logger.router.MODE =
logger.access.MODE = access-file
[log.access-file]
MODE = file
LEVEL = Warn
FILE_NAME = access.log
```
### Set different log levels for different modes
Default logs (>=Warn) goes into `gitea.log`, while Error logs goes into `file-error.log`:
```ini
[log]
LEVEL = Warn
MODE = file, file-error
; by default, the "file" mode will record logs to %(log.ROOT_PATH)/gitea.log, so we don't need to set it
; [log.file]
[log.file-error]
LEVEL = Error
FILE_NAME = file-error.log
```
## Log outputs (mode and writer)
Gitea provides the following log output writers:
- `console` - Log to `stdout` (or `stderr` if it is set in the config)
- `file` - Log to a file
- `conn` - Log to a socket (network or unix)
### Common configuration
Certain configuration is common to all modes of log output:
- `MODE` is the mode of the log output writer. It will default to the mode name in the ini section. Thus `[log.console]` will default to `MODE = console`.
- `LEVEL` is the lowest level that this output will log.
- `STACKTRACE_LEVEL` is the lowest level that this output will print a stacktrace.
- `COLORIZE` will default to `true` for `console` as described, otherwise it will default to `false`.
#### `EXPRESSION`
`EXPRESSION` represents a regular expression that log events must match to be logged by the output writer.
Either the log message, (with colors removed), must match or the `longfilename:linenumber:functionname` must match.
NB: the whole message or string doesn't need to completely match.
Please note this expression will be run in the writer's goroutine but not the logging event goroutine.
#### `FLAGS`
`FLAGS` represents the preceding logging context information that is
printed before each message. It is a comma-separated string set. The order of values does not matter.
It defaults to `stdflags` (= `date,time,medfile,shortfuncname,levelinitial`)
Possible values are:
- `none` or `,` - No flags.
- `date` - the date in the local time zone: `2009/01/23`.
- `time` - the time in the local time zone: `01:23:23`.
- `microseconds` - microsecond resolution: `01:23:23.123123`. Assumes time.
- `longfile` - full file name and line number: `/a/b/c/d.go:23`.
- `shortfile` - final file name element and line number: `d.go:23`.
- `funcname` - function name of the caller: `runtime.Caller()`.
- `shortfuncname` - last part of the function name. Overrides `funcname`.
- `utc` - if date or time is set, use UTC rather than the local time zone.
- `levelinitial` - initial character of the provided level in brackets eg. `[I]` for info.
- `level` - level in brackets `[INFO]`.
- `gopid` - the Goroutine-PID of the context.
- `medfile` - last 20 characters of the filename - equivalent to `shortfile,longfile`.
- `stdflags` - equivalent to `date,time,medfile,shortfuncname,levelinitial`.
### Console mode
In this mode the logger will forward log messages to the stdout and
stderr streams attached to the Gitea process.
For loggers in console mode, `COLORIZE` will default to `true` if not
on windows, or the Windows terminal can be set into ANSI mode or is a
cygwin or Msys pipe.
Settings:
- `STDERR`: **false**: Whether the logger should print to `stderr` instead of `stdout`.
### File mode
In this mode the logger will save log messages to a file.
Settings:
- `FILE_NAME`: The file to write the log events to, relative to `ROOT_PATH`, Default to `%(ROOT_PATH)/gitea.log`. Exception: access log will default to `%(ROOT_PATH)/access.log`.
- `MAX_SIZE_SHIFT`: **28**: Maximum size shift of a single file. 28 represents 256Mb. For details see below.
- `LOG_ROTATE` **true**: Whether to rotate the log files. TODO: if false, will it delete instead on daily rotate, or do nothing?.
- `DAILY_ROTATE`: **true**: Whether to rotate logs daily.
- `MAX_DAYS`: **7**: Delete rotated log files after this number of days.
- `COMPRESS`: **true**: Whether to compress old log files by default with gzip.
- `COMPRESSION_LEVEL`: **-1**: Compression level. For details see below.
`MAX_SIZE_SHIFT` defines the maximum size of a file by left shifting 1 the given number of times (`1 << x`).
The exact behavior at the time of v1.17.3 can be seen [here](https://github.com/go-gitea/gitea/blob/v1.17.3/modules/setting/log.go#L185).
The useful values of `COMPRESSION_LEVEL` are from 1 to (and including) 9, where higher numbers mean better compression.
Beware that better compression might come with higher resource usage.
Must be preceded with a `-` sign.
### Conn mode
In this mode the logger will send log messages over a network socket.
Settings:
- `ADDR`: **:7020**: Sets the address to connect to.
- `PROTOCOL`: **tcp**: Set the protocol, either "tcp", "unix" or "udp".
- `RECONNECT`: **false**: Try to reconnect when connection is lost.
- `RECONNECT_ON_MSG`: **false**: Reconnect host for every single message.
### The "Router" logger
The Router logger logs the following message types when Gitea's route handlers work:
- `started` messages will be logged at TRACE level
- `polling`/`completed` routers will be logged at INFO. Exception: "/assets" static resource requests are also logged at TRACE.
- `slow` routers will be logged at WARN
- `failed` routers will be logged at WARN
### The "XORM" logger
To make XORM outputs SQL logs, the `LOG_SQL` in `[database]` section should also be set to `true`.
### The "Access" logger
The Access logger is a new logger since Gitea 1.9. It provides a NCSA
Common Log compliant log format. It's highly configurable but caution
should be taken when changing its template. The main benefit of this
logger is that Gitea can now log accesses in a standard log format so
standard tools may be used.
You can enable this logger using `logger.access.MODE = ...`.
If desired the format of the Access logger can be changed by changing
the value of the `ACCESS_LOG_TEMPLATE`.
Please note, the access logger will log at `INFO` level, setting the
`LEVEL` of this logger to `WARN` or above will result in no access logs.
#### The ACCESS_LOG_TEMPLATE
This value represents a go template. Its default value is
```tmpl
{{.Ctx.RemoteHost}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}" "{{.Ctx.Req.UserAgent}}"`
```
The template is passed following options:
- `Ctx` is the `context.Context`
- `Identity` is the `SignedUserName` or `"-"` if the user is not logged in
- `Start` is the start time of the request
- `ResponseWriter` is the `http.ResponseWriter`
Caution must be taken when changing this template as it runs outside of
the standard panic recovery trap. The template should also be as simple
as it runs for every request.
## Releasing-and-Reopening, Pausing and Resuming logging
If you are running on Unix you may wish to release-and-reopen logs in order to use `logrotate` or other tools.
It is possible force Gitea to release and reopen it's logging files and connections by sending `SIGUSR1` to the
running process, or running `gitea manager logging release-and-reopen`.
Alternatively, you may wish to pause and resume logging - this can be accomplished through the use of the
`gitea manager logging pause` and `gitea manager logging resume` commands. Please note that whilst logging
is paused log events below INFO level will not be stored and only a limited number of events will be stored.
Logging may block, albeit temporarily, slowing Gitea considerably whilst paused - therefore it is
recommended that pausing only done for a very short period of time.
## Adding and removing logging whilst Gitea is running
It is possible to add and remove logging whilst Gitea is running using the `gitea manager logging add` and `remove` subcommands.
This functionality can only adjust running log systems and cannot be used to start the access or router loggers if they
were not already initialized. If you wish to start these systems you are advised to adjust the app.ini and (gracefully) restart
the Gitea service.
The main intention of these commands is to easily add a temporary logger to investigate problems on running systems where a restart
may cause the issue to disappear.
## Using `logrotate` instead of built-in log rotation
Gitea includes built-in log rotation, which should be enough for most deployments. However, if you instead want to use the `logrotate` utility:
- Disable built-in log rotation by setting `LOG_ROTATE` to `false` in your `app.ini`.
- Install `logrotate`.
- Configure `logrotate` to match your deployment requirements, see `man 8 logrotate` for configuration syntax details.
In the `postrotate/endscript` block send Gitea a `USR1` signal via `kill -USR1` or `kill -10` to the `gitea` process itself,
or run `gitea manager logging release-and-reopen` (with the appropriate environment).
Ensure that your configurations apply to all files emitted by Gitea loggers as described in the above sections.
- Always do `logrotate /etc/logrotate.conf --debug` to test your configurations.
- If you are using docker and are running from outside the container you can use
`docker exec -u $OS_USER $CONTAINER_NAME sh -c 'gitea manager logging release-and-reopen'`
or `docker exec $CONTAINER_NAME sh -c '/bin/s6-svc -1 /etc/s6/gitea/'` or send `USR1` directly to the Gitea process itself.
The next `logrotate` jobs will include your configurations, so no restart is needed.
You can also immediately reload `logrotate` with `logrotate /etc/logrotate.conf --force`.

View File

@ -1,524 +0,0 @@
---
date: "2019-04-02T17:06:00+01:00"
title: "Logging Configuration"
slug: "logging-configuration"
weight: 40
toc: false
draft: false
aliases:
- /en-us/logging-configuration
menu:
sidebar:
parent: "administration"
name: "Logging Configuration"
weight: 40
identifier: "logging-configuration"
---
# Logging Configuration
The logging configuration of Gitea mainly consists of 3 types of components:
- The `[log]` section for general configuration
- `[log.<sublogger>]` sections for the configuration of different log outputs
- `[log.<sublogger>.<group>]` sections for output specific configuration of a log group
As mentioned below, there is a fully functional log output by default, so it is not necessary to define one.
**Table of Contents**
{{< toc >}}
## Collecting Logs for Help
To collect logs for help and issue report, see [Support Options]({{< relref "doc/help/support.en-us.md" >}}).
## The `[log]` section
Configuration of logging facilities in Gitea happen in the `[log]` section and it's subsections.
In the top level `[log]` section the following configurations can be placed:
- `ROOT_PATH`: (Default: **%(GITEA_WORK_DIR)/log**): Base path for log files
- `MODE`: (Default: **console**) List of log outputs to use for the Default logger.
- `ROUTER`: (Default: **console**): List of log outputs to use for the Router logger.
- `ACCESS`: List of log outputs to use for the Access logger.
- `XORM`: (Default: **,**) List of log outputs to use for the XORM logger.
- `ENABLE_ACCESS_LOG`: (Default: **false**): whether the Access logger is allowed to emit logs
- `ENABLE_XORM_LOG`: (Default: **true**): whether the XORM logger is allowed to emit logs
For details on the loggers check the "Log Groups" section.
Important: log outputs won't be used if you don't enable them for the desired loggers in the corresponding list value.
Lists are specified as comma separated values. This format also works in subsection.
This section may be used for defining default values for subsections.
Examples:
- `LEVEL`: (Default: **Info**) Least severe log events to persist. Case insensitive. The full list of levels as of v1.17.3 can be read [here](https://github.com/go-gitea/gitea/blob/v1.17.3/custom/conf/app.example.ini#L507).
- `STACKTRACE_LEVEL`: (Default: **None**) For this and more severe events the stacktrace will be printed upon getting logged.
Some values are not inherited by subsections. For details see the "Non-inherited default values" section.
## Log outputs
Log outputs are the targets to which log messages will be sent.
The content and the format of the log messages to be saved can be configured in these.
Log outputs are also called subloggers.
Gitea provides 4 possible log outputs:
- `console` - Log to `os.Stdout` or `os.Stderr`
- `file` - Log to a file
- `conn` - Log to a socket (network or unix)
- `smtp` - Log via email
By default, Gitea has a `console` output configured, which is used by the loggers as seen in the section "The log section" above.
### Common configuration
Certain configuration is common to all modes of log output:
- `MODE` is the mode of the log output. It will default to the sublogger
name, thus `[log.console.router]` will default to `MODE = console`.
For mode specific confgurations read further.
- `LEVEL` is the lowest level that this output will log. This value
is inherited from `[log]` and in the case of the non-default loggers
from `[log.sublogger]`.
- `STACKTRACE_LEVEL` is the lowest level that this output will print
a stacktrace. This value is inherited.
- `COLORIZE` will default to `true` for `console` as
described, otherwise it will default to `false`.
### Non-inherited default values
There are several values which are not inherited as described above but
rather default to those specific to type of logger, these are:
`EXPRESSION`, `FLAGS`, `PREFIX` and `FILE_NAME`.
#### `EXPRESSION`
`EXPRESSION` represents a regular expression that log events must match to be logged by the sublogger. Either the log message, (with colors removed), must match or the `longfilename:linenumber:functionname` must match. NB: the whole message or string doesn't need to completely match.
Please note this expression will be run in the sublogger's goroutine
not the logging event subroutine. Therefore it can be complicated.
#### `FLAGS`
`FLAGS` represents the preceding logging context information that is
printed before each message. It is a comma-separated string set. The order of values does not matter.
Possible values are:
- `none` or `,` - No flags.
- `date` - the date in the local time zone: `2009/01/23`.
- `time` - the time in the local time zone: `01:23:23`.
- `microseconds` - microsecond resolution: `01:23:23.123123`. Assumes
time.
- `longfile` - full file name and line number: `/a/b/c/d.go:23`.
- `shortfile` - final file name element and line number: `d.go:23`.
- `funcname` - function name of the caller: `runtime.Caller()`.
- `shortfuncname` - last part of the function name. Overrides
`funcname`.
- `utc` - if date or time is set, use UTC rather than the local time
zone.
- `levelinitial` - Initial character of the provided level in brackets eg. `[I]` for info.
- `level` - Provided level in brackets `[INFO]`
- `medfile` - Last 20 characters of the filename - equivalent to
`shortfile,longfile`.
- `stdflags` - Equivalent to `date,time,medfile,shortfuncname,levelinitial`
### Console mode
In this mode the logger will forward log messages to the stdout and
stderr streams attached to the Gitea process.
For loggers in console mode, `COLORIZE` will default to `true` if not
on windows, or the windows terminal can be set into ANSI mode or is a
cygwin or Msys pipe.
Settings:
- `STDERR`: **false**: Whether the logger should print to `stderr` instead of `stdout`.
### File mode
In this mode the logger will save log messages to a file.
Settings:
- `FILE_NAME`: The file to write the log events to. For details see below.
- `MAX_SIZE_SHIFT`: **28**: Maximum size shift of a single file. 28 represents 256Mb. For details see below.
- `LOG_ROTATE` **true**: Whether to rotate the log files. TODO: if false, will it delete instead on daily rotate, or do nothing?.
- `DAILY_ROTATE`: **true**: Whether to rotate logs daily.
- `MAX_DAYS`: **7**: Delete rotated log files after this number of days.
- `COMPRESS`: **true**: Whether to compress old log files by default with gzip.
- `COMPRESSION_LEVEL`: **-1**: Compression level. For details see below.
The default value of `FILE_NAME` depends on the respective logger facility.
If unset, their own default will be used.
If set it will be relative to the provided `ROOT_PATH` in the master `[log]` section.
`MAX_SIZE_SHIFT` defines the maximum size of a file by left shifting 1 the given number of times (`1 << x`).
The exact behavior at the time of v1.17.3 can be seen [here](https://github.com/go-gitea/gitea/blob/v1.17.3/modules/setting/log.go#L185).
The useful values of `COMPRESSION_LEVEL` are from 1 to (and including) 9, where higher numbers mean better compression.
Beware that better compression might come with higher resource usage.
Must be preceded with a `-` sign.
### Conn mode
In this mode the logger will send log messages over a network socket.
Settings:
- `ADDR`: **:7020**: Sets the address to connect to.
- `PROTOCOL`: **tcp**: Set the protocol, either "tcp", "unix" or "udp".
- `RECONNECT`: **false**: Try to reconnect when connection is lost.
- `RECONNECT_ON_MSG`: **false**: Reconnect host for every single message.
### SMTP mode
In this mode the logger will send log messages in email.
It is not recommended to use this logger to send general logging
messages. However, you could perhaps set this logger to work on `FATAL` messages only.
Settings:
- `HOST`: **127.0.0.1:25**: The SMTP host to connect to.
- `USER`: User email address to send from.
- `PASSWD`: Password for the smtp server.
- `RECEIVERS`: Email addresses to send to.
- `SUBJECT`: **Diagnostic message from Gitea**. The content of the email's subject field.
## Log Groups
The fundamental thing to be aware of in Gitea is that there are several
log groups:
- The "Default" logger
- The Router logger
- The Access logger
- The XORM logger
There is also the go log logger.
### The go log logger
Go provides its own extremely basic logger in the `log` package,
however, this is not sufficient for our purposes as it does not provide
a way of logging at multiple levels, nor does it provide a good way of
controlling where these logs are logged except through setting of a
writer.
We have therefore redirected this logger to our Default logger, and we
will log anything that is logged using the go logger at the INFO level.
### The "Default" logger
Calls to `log.Info`, `log.Debug`, `log.Error` etc. from the `code.gitea.io/gitea/modules/log` package will log to this logger.
You can configure the outputs of this logger by setting the `MODE`
value in the `[log]` section of the configuration.
Each output sublogger is configured in a separate `[log.sublogger.default]`
which inherits from the sublogger `[log.sublogger]` section and from the
generic `[log]` section, but there are certain default values. These will
not be inherited from the `[log]` section:
- `FLAGS` is `stdflags` (Equal to
`date,time,medfile,shortfuncname,levelinitial`)
- `FILE_NAME` will default to `%(ROOT_PATH)/gitea.log`
- `EXPRESSION` will default to `""`
- `PREFIX` will default to `""`
The provider type of the sublogger can be set using the `MODE` value in
its subsection, but will default to the name. This allows you to have
multiple subloggers that will log to files.
### The "Router" logger
The Router logger has been substantially changed in v1.17. If you are using the router logger for fail2ban or other monitoring
you will need to update this configuration.
You can disable Router log by setting `DISABLE_ROUTER_LOG` or by setting all of its sublogger configurations to `none`.
You can configure the outputs of this
router log by setting the `ROUTER` value in the `[log]` section of the
configuration. `ROUTER` will default to `console` if unset and will default to same level as main logger.
The Router logger logs the following:
- `started` messages will be logged at TRACE level
- `polling`/`completed` routers will be logged at INFO
- `slow` routers will be logged at WARN
- `failed` routers will be logged at WARN
The logging level for the router will default to that of the main configuration. Set `[log.<mode>.router]` `LEVEL` to change this.
Each output sublogger for this logger is configured in
`[log.sublogger.router]` sections. There are certain default values
which will not be inherited from the `[log]` or relevant
`[log.sublogger]` sections:
- `FILE_NAME` will default to `%(ROOT_PATH)/router.log`
- `FLAGS` defaults to `date,time`
- `EXPRESSION` will default to `""`
- `PREFIX` will default to `""`
NB: You can redirect the router logger to send its events to the Gitea
log using the value: `ROUTER = ,`
### The "Access" logger
The Access logger is a new logger for version 1.9. It provides a NCSA
Common Log compliant log format. It's highly configurable but caution
should be taken when changing its template. The main benefit of this
logger is that Gitea can now log accesses in a standard log format so
standard tools may be used.
You can enable this logger using `ENABLE_ACCESS_LOG`. Its outputs are
configured by setting the `ACCESS` value in the `[log]` section of the
configuration. `ACCESS` defaults to `file` if unset.
Each output sublogger for this logger is configured in
`[log.sublogger.access]` sections. There are certain default values
which will not be inherited from the `[log]` or relevant
`[log.sublogger]` sections:
- `FILE_NAME` will default to `%(ROOT_PATH)/access.log`
- `FLAGS` defaults to `` or None
- `EXPRESSION` will default to `""`
- `PREFIX` will default to `""`
If desired the format of the Access logger can be changed by changing
the value of the `ACCESS_LOG_TEMPLATE`.
Please note, the access logger will log at `INFO` level, setting the
`LEVEL` of this logger to `WARN` or above will result in no access logs.
NB: You can redirect the access logger to send its events to the Gitea
log using the value: `ACCESS = ,`
#### The ACCESS_LOG_TEMPLATE
This value represent a go template. It's default value is:
`{{.Ctx.RemoteHost}} - {{.Identity}} {{.Start.Format "[02/Jan/2006:15:04:05 -0700]" }} "{{.Ctx.Req.Method}} {{.Ctx.Req.URL.RequestURI}} {{.Ctx.Req.Proto}}" {{.ResponseWriter.Status}} {{.ResponseWriter.Size}} "{{.Ctx.Req.Referer}}" "{{.Ctx.Req.UserAgent}}"`
The template is passed following options:
- `Ctx` is the `context.Context`
- `Identity` is the `SignedUserName` or `"-"` if the user is not logged
in
- `Start` is the start time of the request
- `ResponseWriter` is the `http.ResponseWriter`
Caution must be taken when changing this template as it runs outside of
the standard panic recovery trap. The template should also be as simple
as it runs for every request.
### The "XORM" logger
The XORM logger is a long-standing logger that exists to collect XORM
log events. It is enabled by default but can be switched off by setting
`ENABLE_XORM_LOG` to `false` in the `[log]` section. Its outputs are
configured by setting the `XORM` value in the `[log]` section of the
configuration. `XORM` defaults to `,` if unset, meaning it is redirected
to the main Gitea log.
XORM will log SQL events by default. This can be changed by setting
the `LOG_SQL` value to `false` in the `[database]` section.
Each output sublogger for this logger is configured in
`[log.sublogger.xorm]` sections. There are certain default values
which will not be inherited from the `[log]` or relevant
`[log.sublogger]` sections:
- `FILE_NAME` will default to `%(ROOT_PATH)/xorm.log`
- `FLAGS` defaults to `date,time`
- `EXPRESSION` will default to `""`
- `PREFIX` will default to `""`
## Debugging problems
When submitting logs in Gitea issues it is often helpful to submit
merged logs obtained by either by redirecting the console log to a file or
copying and pasting it. To that end it is recommended to set your logging to:
```ini
[database]
LOG_SQL = false ; SQL logs are rarely helpful unless we specifically ask for them
...
[log]
MODE = console
LEVEL = debug ; please set the level to debug when we are debugging a problem
ROUTER = console
COLORIZE = false ; this can be true if you can strip out the ansi coloring
ENABLE_SSH_LOG = true ; shows logs related to git over SSH.
```
Sometimes it will be helpful get some specific `TRACE` level logging restricted
to messages that match a specific `EXPRESSION`. Adjusting the `MODE` in the
`[log]` section to `MODE = console,traceconsole` to add a new logger output
`traceconsole` and then adding its corresponding section would be helpful:
```ini
[log.traceconsole] ; traceconsole here is just a name
MODE = console ; this is the output that the traceconsole writes to
LEVEL = trace
EXPRESSION = ; putting a string here will restrict this logger to logging only those messages that match this expression
```
(It's worth noting that log messages that match the expression at or above debug
level will get logged twice so don't worry about that.)
`STACKTRACE_LEVEL` should generally be left unconfigured (and hence kept at
`none`). There are only very specific occasions when it useful.
## Empty Configuration
The empty configuration is equivalent to:
```ini
[log]
ROOT_PATH = %(GITEA_WORK_DIR)/log
MODE = console
LEVEL = Info
STACKTRACE_LEVEL = None
ENABLE_ACCESS_LOG = false
ENABLE_XORM_LOG = true
XORM = ,
[log.console]
MODE = console
LEVEL = %(LEVEL)
STACKTRACE_LEVEL = %(STACKTRACE_LEVEL)
FLAGS = stdflags
PREFIX =
COLORIZE = true # Or false if your windows terminal cannot color
```
This is equivalent to sending all logs to the console, with default go log being sent to the console log too.
## Releasing-and-Reopening, Pausing and Resuming logging
If you are running on Unix you may wish to release-and-reopen logs in order to use `logrotate` or other tools.
It is possible force Gitea to release and reopen it's logging files and connections by sending `SIGUSR1` to the
running process, or running `gitea manager logging release-and-reopen`.
Alternatively, you may wish to pause and resume logging - this can be accomplished through the use of the
`gitea manager logging pause` and `gitea manager logging resume` commands. Please note that whilst logging
is paused log events below INFO level will not be stored and only a limited number of events will be stored.
Logging may block, albeit temporarily, slowing Gitea considerably whilst paused - therefore it is
recommended that pausing only done for a very short period of time.
## Adding and removing logging whilst Gitea is running
It is possible to add and remove logging whilst Gitea is running using the `gitea manager logging add` and `remove` subcommands.
This functionality can only adjust running log systems and cannot be used to start the access or router loggers if they
were not already initialized. If you wish to start these systems you are advised to adjust the app.ini and (gracefully) restart
the Gitea service.
The main intention of these commands is to easily add a temporary logger to investigate problems on running systems where a restart
may cause the issue to disappear.
## Log colorization
Logs to the console will be colorized by default when not running on
Windows. Terminal sniffing will occur on Windows and if it is
determined that we are running on a terminal capable of color we will
colorize.
Further, on \*nix it is becoming common to have file logs that are
colored by default. Therefore file logs will be colorised by default
when not running on Windows.
You can switch on or off colorization by using the `COLORIZE` value.
From a development point of view. If you write
`log.Info("A %s string", "formatted")` the `formatted` part of the log
message will be Bolded on colorized logs.
You can change this by either rendering the formatted string yourself.
Or you can wrap the value in a `log.ColoredValue` struct.
The `log.ColoredValue` struct contains a pointer to value, a pointer to
string of bytes which should represent a color and second set of reset
bytes. Pointers were chosen to prevent copying of large numbers of
values. There are several helper methods:
- `log.NewColoredValue` takes a value and 0 or more color attributes
that represent the color. If 0 are provided it will default to a cached
bold. Note, it is recommended that color bytes constructed from
attributes should be cached if this is a commonly used log message.
- `log.NewColoredValuePointer` takes a pointer to a value, and
0 or more color attributes that represent the color.
- `log.NewColoredValueBytes` takes a value and a pointer to an array
of bytes representing the color.
These functions will not double wrap a `log.ColoredValue`. They will
also set the `resetBytes` to the cached `resetBytes`.
The `colorBytes` and `resetBytes` are not exported to prevent
accidental overwriting of internal values.
## ColorFormat & ColorFormatted
Structs may implement the `log.ColorFormatted` interface by implementing the `ColorFormat(fmt.State)` function.
If a `log.ColorFormatted` struct is logged with `%-v` format, its `ColorFormat` will be used instead of the usual `%v`. The full `fmt.State` will be passed to allow implementers to look at additional flags.
In order to help implementers provide `ColorFormat` methods. There is a
`log.ColorFprintf(...)` function in the log module that will wrap values in `log.ColoredValue` and recognise `%-v`.
In general it is recommended not to make the results of this function too verbose to help increase its versatility. Usually this should simply be an `ID`:`Name`. If you wish to make a more verbose result, it is recommended to use `%-+v` as your marker.
## Log Spoofing protection
In order to protect the logs from being spoofed with cleverly
constructed messages. Newlines are now prefixed with a tab and control
characters except those used in an ANSI CSI are escaped with a
preceding `\` and their octal value.
## Creating a new named logger group
Should a developer wish to create a new named logger, `NEWONE`. It is
recommended to add an `ENABLE_NEWONE_LOG` value to the `[log]`
section, and to add a new `NEWONE` value for the modes.
A function like `func newNewOneLogService()` is recommended to manage
construction of the named logger. e.g.
```go
func newNewoneLogService() {
EnableNewoneLog = Cfg.Section("log").Key("ENABLE_NEWONE_LOG").MustBool(false)
Cfg.Section("log").Key("NEWONE").MustString("file") // or console? or "," if you want to send this to default logger by default
if EnableNewoneLog {
options := newDefaultLogOptions()
options.filename = filepath.Join(LogRootPath, "newone.log")