out_file:Inconsistent path settings and symlink_path settings. #4490
Description
Describe the bug
If tag is set in the chunk key, We will get an error unless we specify the tag in the path setting of the out_file plugin.
However, there is no such check in the symlink_path setting. (i.e., no error occurs even if no tag is specified.)
Also, this check is a setting to make the pass unique, but if we use a part of tag such as tag[0] instead of tag, no error will occur.
Is this an intended spec? Or is there a check omitted?
To Reproduce
For example, use the following input.
<source>
@type sample
sample {"message":"hoge"}
tag test.hoge
</source>
<source>
@type sample
sample {"message":"fuga"}
tag test.fuga
</source>At this time, setting the out_file plugin as follows will result in an error.
<match test.**>
@type file
path /tmp/test
symlink_path /tmp/test/link/${tag}
append true
<buffer time,tag>
@type file
path /tmp/test/buffer
timekey 60
timekey_wait 3
</buffer>
</match> $ bundle exec fluentd -c /Users/fluentd/configuration.txt
2024-04-26 10:20:51 +0900 [info]: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2024-04-26 10:20:51 +0900 [info]: parsing config file is succeeded path="/Users/fluentd/configuration.txt"
2024-04-26 10:20:51 +0900 [info]: gem 'fluentd' version '1.16.5'
2024-04-26 10:20:51 +0900 [error]: config error file="/Users/fluentd/configuration.txt" error_class=Fluent::ConfigError error="Parameter 'path: /tmp/test/test.%Y%m%d%H%M.log' doesn't have tag placeholder"
However, if the out_file plugin is set up as follows, there is no error and two outputs compete for a single path.
<match test.**>
@type file
path /tmp/test/${tag[0]}
symlink_path /tmp/test/link/${tag}
append true
<buffer time,tag>
@type file
path /tmp/test/buffer/tag
timekey 60
timekey_wait 3
</buffer>
</match>/tmp/test $ ls -R
buffer test.202405091047.log test.202405091049.log
link test.202405091048.log
./buffer:
tag
./buffer/tag:
buffer.b617fba30cfcdcb64c554db644d9ef7c5.log
buffer.b617fba30cfcdcb64c554db644d9ef7c5.log.meta
buffer.b617fba30dd0f05a6f9a793e93f657172.log
buffer.b617fba30dd0f05a6f9a793e93f657172.log.meta
./link:
test test.fuga test.hoge
/tmp/test $ tail test.202405091048.log
2024-05-09T10:48:00+09:00 test.fuga {"message":"fuga"}
2024-05-09T10:48:01+09:00 test.fuga {"message":"fuga"}
2024-05-09T10:48:02+09:00 test.fuga {"message":"fuga"}
...
2024-05-09T10:48:59+09:00 test.fuga {"message":"fuga"}
2024-05-09T10:48:00+09:00 test.hoge {"message":"hoge"}
2024-05-09T10:48:01+09:00 test.hoge {"message":"hoge"}
...
2024-05-09T10:48:59+09:00 test.hoge {"message":"hoge"}
Furthermore, if the out_file plugin is set up as follows, there will be no error and the two will compete for a single symlink_path.
<match test.**>
@type file
path /tmp/test/${tag[0]}
symlink_path /tmp/test/link
append true
<buffer time,tag>
@type file
path /tmp/test/buffer
timekey 60
timekey_wait 3
</buffer>
</match>$ ls -R
buffer link
./buffer:
buffer.b616f84e16b5e9575a8a61ca0cbf34536.log buffer.b616f84e16b968e52f8d0c2bc37bdb9d9.log
buffer.b616f84e16b5e9575a8a61ca0cbf34536.log.meta buffer.b616f84e16b968e52f8d0c2bc37bdb9d9.log.meta
./link:
test
/tmp/test $ tail link/test
2024-04-26T13:26:39+09:00 test.fuga {"message":"fuga"}
2024-04-26T13:26:40+09:00 test.fuga {"message":"fuga"}
2024-04-26T13:26:41+09:00 test.fuga {"message":"fuga"}
...
/tmp/test $ tail link/test
2024-04-26T13:27:00+09:00 test.hoge {"message":"hoge"}
2024-04-26T13:27:01+09:00 test.hoge {"message":"hoge"}
2024-04-26T13:27:02+09:00 test.hoge {"message":"hoge"}
...
Expected behavior
In either case, an error should occur. (Or this is the intended specification.)
Your Environment
- Fluentd version:1.17.0
- Operating system:
ProductName: macOS
ProductVersion: 13.0
BuildVersion: 22A380
- Kernel version:22.1.0Your Configuration
See To Reproduce section.
Your Error Log
See To Reproduce section.
Additional context
No response