Skip to content
Merged
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
## ChangeLog

[All published release notes](https://github.com/Cacti/plugin_syslog/blob/develop/docs/RELEASE_HISTORY.md)

[Version 5.0 release notes and upgrade guidance](https://github.com/Cacti/plugin_syslog/blob/develop/docs/RELEASE_NOTES_5.0.md) · [Contributors and change history](https://github.com/Cacti/plugin_syslog/blob/develop/docs/COMMITS_5.0.md)

--- develop ---

* docs: Refresh README release/source compatibility, installation guidance and source-checked behavior; add published release notes and 5.0 upgrade guidance
* bug: Keep log column resizing within the results area on both log tabs, preserve message ellipsis, and use an accessible Font Awesome export button
* feature: Standardize both schema and partition CLI help with Cacti-style version banners and options, separate maintenance warnings, and replace the unreleased millisecond converter with --convert-ms
* feature: Add a data-preserving partition rebuild CLI with optional --convert-ms, verified 10,000-row copies, interrupted-copy recovery, and database-size, extra-disk-space and several-hour maintenance warnings
Expand Down
148 changes: 67 additions & 81 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,8 @@
# syslog

The syslog plugin is a Cacti plugin that has been around for more than a decade.
It was inspired by the 'aloe' and 'h.aloe' plugins originally developed by the
Cacti users sidewinder and Harlequin in the early 2000's. As you will be able
to see from the ChangeLog, it has undergone several changes throughout the
years, and remains, even today when you have enterprise offering from both
Elastic and Splunk, remains a relevant plugin for small to medium sized
companies.

It provides a simple Syslog event search an Alert generation and notification
interface that can generate both HTML and SMS messages for operations personnel
who wish to receive notifications inside of a data or network operations center.

When combined by the Linux SNMPTT package, it can be converted into an SNMP Trap
and Inform receiver and notification engine as the SNMPTT tool will receive SNMP
Traps and Informs and convert them into Syslog messages on your log server.
These syslog messages can then be consumed by the syslog plugin. So, this tool
is quite handy.
Syslog collects and searches log messages inside Cacti, applies alert and removal
rules, and sends notifications or runs configured commands when rules match.
It can also consume SNMP traps and informs forwarded as Syslog messages by SNMPTT.

Incoming program names have leading and trailing parentheses and square brackets
removed before reference lookup and rule processing. Interior punctuation is
Expand All @@ -42,6 +28,30 @@ without scanning or updating historical message rows. Existing historical flags
are not converted into active breaches. See [threshold recovery](docs/THRESHOLD_RECOVERY.md)
for lifecycle and deployment details.

## Versions and compatibility

This README describes the `develop` source branch.

| Code | Plugin version | Minimum Cacti declared in `INFO` |
| --- | --- | --- |
| [Source branch](https://github.com/Cacti/plugin_syslog/blob/develop/INFO) | 5.0 | 1.2.23 |
| [Published v4.2](https://github.com/Cacti/plugin_syslog/releases/tag/v4.2) | 4.2 | 1.2.23 |

Use the published release’s assets and tagged documentation when installing that
package. Branch code can include later changes, even when its version number
matches the release. Cacti’s own PHP and database requirements also apply.

Install this plugin in **`<cacti-root>/plugins/syslog`**, using the directory
name `syslog` rather than `plugin_syslog`.

See [CHANGELOG.md](https://github.com/Cacti/plugin_syslog/blob/develop/CHANGELOG.md) for the version history.

[All published release notes](https://github.com/Cacti/plugin_syslog/blob/develop/docs/RELEASE_HISTORY.md)

[Version 5.0 release notes and upgrade guidance](https://github.com/Cacti/plugin_syslog/blob/develop/docs/RELEASE_NOTES_5.0.md) · [Contributors and change history](https://github.com/Cacti/plugin_syslog/blob/develop/docs/COMMITS_5.0.md)

[Setup module ownership](https://github.com/Cacti/plugin_syslog/blob/develop/docs/setup-modules.md) describes how `setup.php` and the `includes/` modules divide the plugin code.

## Core Features

* Message filtering
Expand Down Expand Up @@ -337,10 +347,11 @@ item yet.

The plugin declares `compat = 1.2.23` in `INFO`, so it needs Cacti 1.2.23 or
later, and PHP 8.0 or later. Every non-test PHP file passes `php -l` on PHP 8.0
and 8.1. The Linux integration workflow runs PHP 8.2, 8.3, and 8.4 against the
Cacti `1.2.x` branch on MariaDB 11.8, including installation, plugin enablement,
polling, and sample Syslog processing. Other PHP, Cacti, and database versions
are not exercised by CI.
and 8.1. The [Linux integration workflow](https://github.com/Cacti/plugin_syslog/blob/develop/.github/workflows/plugin-ci-workflow.yml)
runs PHP 8.2 through 8.5 against Cacti `1.2.x` on MariaDB 11.8, including
installation, plugin enablement, polling, and sample Syslog processing. PHP 8.6
is a non-blocking experimental job. Use the PHP version required by your Cacti
release even when the plugin's syntax supports an older interpreter.

This plugin does not ship its own `composer.json`. Composer validation and
dependency installation in CI use the manifest supplied by the checked-out
Expand Down Expand Up @@ -416,8 +427,8 @@ run against Cacti's own Composer-managed vendor tree, plus a disposable
Docker/Playwright end-to-end harness. GitHub CI runs a single integration-test
workflow that checks out the Cacti `1.2.x` branch, runs the Pest suite, the
regression scripts, and the changed-line coverage check, and exercises the
syslog poller end-to-end on the PHP 8.2-8.4 matrix on Linux. The
Docker E2E runner refuses unsafe temporary-directory paths before performing
syslog poller end-to-end on Linux. PHP 8.2 through 8.5 are the regular matrix;
PHP 8.6 is experimental. The Docker E2E runner refuses unsafe temporary-directory paths before performing
recursive cleanup.

## Installation
Expand Down Expand Up @@ -451,10 +462,11 @@ Upgrade Advisor when it finds existing syslog tables. Its upgrade type is
change how a legacy non-partitioned `syslog` table is converted. An existing
partitioned install keeps its messages, rules, and hosts with each choice.

Note: You must rename config.php.dist in the syslog plugin directory to
config.php and make changes there for the location of the database, user,
password, and host. This is especially important if you are using a remote
logging database server.
Before installing, copy `config.php.dist` to `config.php` and configure the
database connection. Preserve this file during upgrades. A collector with an
independent Syslog database uses `config_local.php.dist` as the template for
`config_local.php`; when both configuration files exist, `config_local.php`
takes precedence.

To upgrade an existing install, replacing the files is not enough: follow
[Important: Upgrading the Plugin](#important-upgrading-the-plugin), which
Expand Down Expand Up @@ -529,40 +541,24 @@ $template cacti_syslog,"INSERT INTO syslog_incoming(facility_id, priority_id, pr
*.* >localhost,my_database,my_user,my_password;cacti_syslog
```

For version below 2.0 it should be:

```console
$ModLoad imudp
$UDPServerRun 514
$ModLoad ommysql

$template cacti_syslog,"INSERT INTO syslog_incoming(facility_id, priority_id, program, logtime, host, message)
values (%syslogfacility%, %syslogpriority%, '%programname%', '%timegenerated:::date-mysql%', '%HOSTNAME%', TRIM('%msg%'))", SQL
The template above targets the current `logtime` schema. Examples for
pre-2.5 releases used a different incoming schema and must not be copied into a
current install. See the README at the matching tag for a legacy migration.

. >localhost,my_database,my_user,my_password;cacti_syslog
```

For version 2.4 and above it should be:
Install your distribution's rsyslog MySQL output module, replace the database
and credential placeholders, then validate and restart rsyslog:

```console
module(load="ommysql")
action(type="ommysql" server="localhost" serverport="3306"
db="syslogDB" uid="userID" pwd="passwdID")

$template cacti_syslog,"INSERT INTO syslog_incoming(facility_id, priority_id, program, logtime, host, message)
values (%syslogfacility%, %syslogpriority%, '%programname%', '%timegenerated:::date-mysql%', '%HOSTNAME%', TRIM('%msg%'))", SQL
```

For CentOS/RHEL systems you will all need to install the rsyslog-mysql package

```
yum install rsyslog-mysql
rsyslogd -N1
systemctl restart rsyslog
```

If you are upgrading to version 2.5 from an earlier version, make sure that you
update this template format and restart rsyslog. You may loose some syslog
data, but doing this in a timely fashion, will minimize data loss.
Do not load a module or listener twice if another configuration file already
loads it. A modern `ommysql` action must explicitly select the Cacti template;
its default template does not target `syslog_incoming`. See the
[rsyslog module reference](https://docs.rsyslog.com/doc/configuration/modules/ommysql.html)
for current configuration options. Protect the configuration file because it
contains database credentials.

Senders that only have an IPv6 address are supported. With hostname resolution
enabled the plugin accepts an IPv4 or IPv6 address as it is, and stores an IPv6
Expand All @@ -574,7 +570,7 @@ address.
Ensure you restart rsyslog after these changes are completed. Other logging
servers such as Syslog-NG are also supported with this plugin. Please see some
additional documentation here: [Cacti Documentation
Site](https://docs.cacti.net/plugin:syslog.config)
Site](https://github.com/Cacti/documentation/blob/develop/Plugin-Syslog.md)

We are using the pure integer values that rsyslog provides to both the priority
and facility in this version syslog, which makes the data collection must less
Expand All @@ -585,35 +581,25 @@ To setup log forwarding from your network switches and routers, and from your
various Linux, UNIX, and other operating system devices, please see their
respective documentation.

Finally, it's important, especially in more recent versions of MySQL and MariaDB
to set a correct SQL Mode. These more recent SQL's prevent certain previously
allowable syntax such as an empty data and certain group by limitations in the
SQL itself. Therefore, you need to ensure that the SQL mode of the database is
correct. To do this, first start by editing either `/etc/my.cnf` or
`/etc/my.cnf.d/server.cnf` and inserting the SQL mode variable into the database
configuration. For example:

```ini
[mysqld]
sql_mode=NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER
```
### Database SQL mode and grouping

### Special Note for the Grouping feature
Syslog now supports the ability to group similar messages from a specific host together
for easy summarization for larger systems you may need to expand the default mariadb group_concat_max_len
The default is 1MB you may need to raise it much higher depending on your needs

After this change, you should log into the mysql server and run the following
command:
Older instructions set `sql_mode=NO_ENGINE_SUBSTITUTION,NO_AUTO_CREATE_USER`.
Do not copy that setting into a current MySQL configuration:
[MySQL removed `NO_AUTO_CREATE_USER` in 8.0](https://dev.mysql.com/doc/refman/8.0/en/upgrading-from-previous-series.html).
Inspect the actual modes when diagnosing a schema or migration failure:

```console
mysql> show global variables like 'sql_mode';
```sql
SELECT @@GLOBAL.sql_mode, @@SESSION.sql_mode;
```

And ensure that it matches the setting that you placed in the database
configuration. If it does not, please search for the configuration that is
making this SQL mode other than what you required. More recent versions of
MySQL and MariaDB will source multiple database configuration files.
Follow the migration guidance for your database and plugin release rather than
changing the server-wide modes for all applications.

### Special Note for the Grouping feature

Syslog can group similar messages from a host. For large groups, inspect
`group_concat_max_len` and raise it only as required by the query output and
available memory; its default depends on the database product and version.

## Tables

Expand Down
25 changes: 25 additions & 0 deletions docs/COMMITS_5.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Syslog 5.0 contributors and change history

[Release notes](RELEASE_NOTES_5.0.md) · [Compare changes](https://github.com/Cacti/plugin_syslog/compare/v4.2...2489d3d73555ac8b914e5768da55f1f15621b97c)

Unreleased snapshot: `v4.2..2489d3d73555ac8b914e5768da55f1f15621b97c`. Refresh this comparison before publishing 5.0.

## Contributors

Thank you to the contributors whose work is recorded in this range:

[@aqr199](https://github.com/aqr199), [@arno-st](https://github.com/arno-st), [@bmfmancini](https://github.com/bmfmancini), [@cigamit](https://github.com/cigamit), [@EkaterinePapava](https://github.com/EkaterinePapava), [@hljtql](https://github.com/hljtql), [@ka1lie](https://github.com/ka1lie), [@Linegod](https://github.com/Linegod), [@mixneko](https://github.com/mixneko), [@nakayama1869](https://github.com/nakayama1869), [@somethingwithproof](https://github.com/somethingwithproof), [@TheWitness](https://github.com/TheWitness), [@weblate](https://github.com/weblate), [@xmacan](https://github.com/xmacan), [@yeager](https://github.com/yeager).

Additional recorded author names are retained in the complete comparison when GitHub does not associate them with an account.

[Browse the paginated commit history](https://github.com/Cacti/plugin_syslog/commits/2489d3d73555ac8b914e5768da55f1f15621b97c).
Commit pages retain the original author and co-author records, including merge,
translation and automation history.

## Selected changes

- [3776f9e8af](https://github.com/Cacti/plugin_syslog/commit/3776f9e8af90010fbefdc65d25dbb1df99731d39) Add dashboards for log investigations; by [@bmfmancini](https://github.com/bmfmancini).
- [c9614425c9](https://github.com/Cacti/plugin_syslog/commit/c9614425c9d39a6aede9641cd26858b9c5151854) Add parallel Syslog processing; by [@bmfmancini](https://github.com/bmfmancini).
- [c21ebb8d3a](https://github.com/Cacti/plugin_syslog/commit/c21ebb8d3aa9c0fab930f605ab6ab42db48a8c29) Share dashboard card controls with the Status view; by [@TheWitness](https://github.com/TheWitness).

Use the comparison and paginated history to inspect other changes and original attribution.
18 changes: 18 additions & 0 deletions docs/RELEASE_HISTORY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Syslog release history

Release notes for every published GitHub release, with version-specific highlights, compatibility requirements, upgrade checks and linked contributor credits. Historical notes preserve the original detailed announcements.

The [5.0 development notes](RELEASE_NOTES_5.0.md) describe an unreleased version and are separate from this published-release history.

- [v4.2](RELEASE_NOTES_4.2.md)
- [v4.1](releases/v4.1.md)
- [v3.1](releases/v3.1.md)
- [v3.0](releases/v3.0.md)
- [v2.9](releases/v2.9.md)
- [v2.8](releases/v2.8.md)
- [v2.7](releases/v2.7.md)
- [v2.6](releases/v2.6.md)
- [v2.5](releases/v2.5.md)
- [2.3](releases/2.3.md)
- [v2.2](releases/v2.2.md)
- [v2.1](releases/v2.1.md)
51 changes: 51 additions & 0 deletions docs/RELEASE_NOTES_4.2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Syslog 4.2: fixes for rule editing, filtering and message cleanup

Syslog 4.2 is a maintenance release focused on everyday log management: creating
program-based alerts, finding devices in the filter, and removing messages. It
also includes PHP 8.2 deprecation fixes and updated rsyslog configuration guidance.

## What changes for operators

- **Program-based alerts:** fixes an error when creating an alert rule based on
a program name ([#220](https://github.com/Cacti/plugin_syslog/issues/220)).
- **Removal expressions:** fixes matching against the `program` column and SQL
errors in removal rules ([#184](https://github.com/Cacti/plugin_syslog/issues/184),
[#217](https://github.com/Cacti/plugin_syslog/issues/217)).
- **Message deletion:** fixes SQL errors when deleting messages
([#218](https://github.com/Cacti/plugin_syslog/issues/218)).
- **Device filtering:** restores devices missing from the Device Filter
([#211](https://github.com/Cacti/plugin_syslog/issues/211)).
- **Deployment fixes:** addresses an `escapeshellarg()` ValueError and PHP 8.2
deprecation messages, and corrects the rsyslog configuration documentation
([#208](https://github.com/Cacti/plugin_syslog/issues/208),
[#216](https://github.com/Cacti/plugin_syslog/issues/216),
[#222](https://github.com/Cacti/plugin_syslog/issues/222)).

## Upgrade and verify

The release declares **Cacti 1.2.23 or later** in its plugin metadata. Back up the
Syslog database, plugin files and local database configuration before replacing
the plugin files. Use the Cacti Plugin Management upgrade path and the
[version 4.2 documentation](https://github.com/Cacti/plugin_syslog/blob/v4.2/README.md#installation)
for the installed version; retain the local database configuration.

After upgrading, confirm that a known message from your normal forwarder reaches
the viewer, that the Device Filter lists the expected devices, and that existing
program-based alert/removal rules handle controlled test messages as intended.
Check the Cacti/PHP logs for SQL and runtime errors. If rollback is necessary,
restore matching plugin files and the database from the pre-upgrade backup.

## Contributors and full history

Thank you to the community members who developed, translated, tested and reviewed
this maintenance release. Contributors recorded between v4.1 and v4.2 include:

[@arno-st](https://github.com/arno-st), [@bmfmancini](https://github.com/bmfmancini), [@click0](https://github.com/click0), [@Coool](https://github.com/Coool), [@edewillians10](https://github.com/edewillians10), [@Harry-Junhua-Huang](https://github.com/Harry-Junhua-Huang), [@ka1lie](https://github.com/ka1lie), [@Linegod](https://github.com/Linegod), [@netniV](https://github.com/netniV), [@TheWitness](https://github.com/TheWitness), [@weblate](https://github.com/weblate), [@xmacan](https://github.com/xmacan), [@xubowin](https://github.com/xubowin), [@zersh01](https://github.com/zersh01).

The history also records contributions under Edwin Lankamp. Unlinked names are kept as recorded; no account is guessed.

[Full changelog](https://github.com/Cacti/plugin_syslog/blob/v4.2/CHANGELOG.md) ·
[All commits and original author links](https://github.com/Cacti/plugin_syslog/compare/v4.1...v4.2)

The separate 5.0 development notes describe an unreleased version; dashboards,
saved searches and other 5.0 features are not part of this 4.2 package.
Loading
Loading