Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@

--- develop ---

* feature: Add manually importable recommendation catalogues for Arista, Dell, HPE, Huawei and NVIDIA in docs/<vendor>.json.gz, built from each vendor's public message references
* feature: Recommendations are matched only against the sending device's vendor (and vendor-neutral records) when the syslog host is a Cacti device whose vendor is recognised from its SNMP sysDescr or device template; other hosts still match every vendor

* feature: Preserve legacy DATETIME logtime history (including TO_DAYS partitioned tables) in cli/upgrade_partitioning.php with --source-time-zone=<zone>. Before any change the zone must be known to PHP/Cacti and the Syslog database, both must agree on its offsets across the stored history, and every value must exist in it. Stored and removed messages are then renamed and copied in verified batches in that zone, and a retry must reuse the same zone
* issue#472: Purge Syslog Devices (Utilities) crashed with "Call to undefined function syslog_db_execute()" because the hook never connected to the Syslog database; it now connects first, reports connection failures, qualifies the Syslog database, and uses NOT EXISTS so a NULL host_id no longer prevents purging
* issue#473: Syslog Status: choosing a card in Add card now appends it (Cacti's select2 dropdowns report changes through jQuery, which the card controller's native listener missed), and the Refresh interval is saved per user through a callback and refreshes the cards in place instead of reloading the page
Expand Down
30 changes: 28 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -789,8 +789,16 @@ which returns JSON.
match the message word as written: `%LINK-3-UPDOWN` matches
`%LINK-3-UPDOWN`, not `LINK-3-UPDOWN`. A recommendation stored as
`LINK-3-UPDOWN` matches only that unprefixed message word.
* All vendors are checked. When several recommendations match, the longest
identifiers are shown first, up to three.
* When the syslog host is a Cacti device whose vendor can be recognised, only
that vendor's recommendations and those with an empty vendor are checked.
The vendor is read from the device's SNMP system description
(`snmp_sysDescr`), or from its device template name when that is empty. The
syslog host is matched to a device by description or hostname. This
prevents short codes such as HPE AOS-CX event numbers from matching the same
number in another vendor's message. The recognised vendors use the names of
the bundled catalogues: Arista, Cisco, Dell, HPE, Huawei, Juniper and NVIDIA.
Otherwise all vendors are checked. When several recommendations match, the
longest identifiers are shown first, up to three.
* Identifiers containing other characters, such as the `#` slot placeholder or
the `|` alternatives in some vendor documentation (`%ASA|PIX-3-331001`), are
stored and exported but can never match. The bundled data has 860 of them;
Expand Down Expand Up @@ -825,6 +833,24 @@ This is not a complete Juniper catalogue and is not automatically installed.
Message templates use bracketed variable names and explanatory wording, not
regular expressions.

Further manually importable catalogues are built from each vendor's public
message references. None of them is installed automatically, and none is
complete. Explanations and actions are condensed from the vendor text; where a
vendor documents no action the field is empty. The `reference` field of each
record links to its source.

| File | Vendor | Records | Coverage and identifier form |
|------|--------|---------|------------------------------|
| `docs/arista.json.gz` | Arista | 1,047 | EOS System Message Guide (EOS 4.15), `%FACILITY-SEV-MNEMONIC` such as `%LINEPROTO-5-UPDOWN`. Later EOS messages are missing. |
| `docs/dell.json.gz` | Dell | 7,142 | PowerEdge iDRAC/CMC messages that can be sent to remote syslog (6,811, MessageID such as `PSU0003`), SmartFabric OS10 (327, `%IFM_ASTATE_UP`) and four OS9 messages (`%CHMGR-2-MAJOR_TEMP`). |
| `docs/hpe.json.gz` | HPE | 4,878 | Comware 5 and 7 switches (`%10IFNET/3/PHY_UPDOWN`, the default log-host format), AOS-CX 10.17 and ArubaOS-Switch 16.10 event IDs (`4701`, `00076`), which have explanations but no actions. |
| `docs/huawei.json.gz` | Huawei | 3,686 | CloudEngine S16700 and 9800/8800/6800 (VRP8) and S1720–S6720 (VRP5) Log References, `%01MODULE/SEV/MNEMONIC` such as `%01IFNET/4/IF_STATE`, as the lookup reads `%%01IFNET/4/IF_STATE(l)`. Devices that send another version number than `01` do not match. AR and NE routers are not included. |
| `docs/nvidia.json.gz` | NVIDIA | 251 | Cumulus Linux 5.18 switchd messages keyed on the function name they contain, FRRouting `[EC nnnnnnnn]` error codes, and the GPU `Xid`/NVSwitch `SXid` families. |

Numeric identifiers, such as HPE event IDs and FRRouting error codes, can match
any number in a message. They are only reliable when the sending device is a
Cacti device whose vendor is recognised (see the matching rules above).

The file picker accepts `.json` and `.json.gz`; pasted JSON is also supported.
Exports (selected rows or the current filters) download as `.json.gz` by default.
Gzip import and export require PHP's zlib extension. Both the compressed file
Expand Down
Binary file added docs/arista.json.gz
Binary file not shown.
Binary file added docs/dell.json.gz
Binary file not shown.
Binary file added docs/hpe.json.gz
Binary file not shown.
Binary file added docs/huawei.json.gz
Binary file not shown.
Binary file added docs/nvidia.json.gz
Binary file not shown.
2 changes: 1 addition & 1 deletion includes/functions.php
Original file line number Diff line number Diff line change
Expand Up @@ -9264,7 +9264,7 @@ function syslog_message_details(string $source, $id, string $logtime = ''): arra
$recommendations = [];

if (function_exists('syslog_recommendations_lookup')) {
$recommendations = syslog_recommendations_format(syslog_recommendations_lookup($record['message']));
$recommendations = syslog_recommendations_format(syslog_recommendations_lookup($record['message'], 3, $record['device']));
}

return [
Expand Down
171 changes: 143 additions & 28 deletions includes/recommendations.php
Original file line number Diff line number Diff line change
Expand Up @@ -520,28 +520,130 @@ function syslog_recommendation_candidates(string $message): array {
return array_map('strval', array_keys($out));
}

/**
* Name the catalogue vendor of a Cacti device from its SNMP sysDescr, falling
* back to its device template name.
*
* The names match the vendor field of the bundled catalogues in docs/.
*
* @param string $sys_descr The device's snmp_sysDescr.
* @param string $template The device template name.
*
* @return string The vendor, or '' when it cannot be determined.
*/
function syslog_recommendation_vendor_from_device(string $sys_descr, string $template = ''): string {
// Network OS before hardware brand: a Dell switch running Cumulus Linux logs
// NVIDIA messages, so NVIDIA is tested first.
$patterns = [
'NVIDIA' => '/\b(?:nvidia|mellanox|cumulus|onyx|mlnx-os)\b/i',
'Arista' => '/\barista\b/i',
'Juniper' => '/\b(?:juniper|junos)\b/i',
'Huawei' => '/\b(?:huawei|vrp)\b/i',
'Dell' => '/\b(?:dell|force10|ftos|powerconnect|powerswitch|idrac|poweredge|os10)\b/i',
'HPE' => '/\b(?:hpe|hp|hewlett[- ]packard|comware|procurve|aruba|aos-cx|arubaos)\b/i',
'Cisco' => '/\b(?:cisco|nx-os|ios-xe|ios-xr)\b/i'
];

foreach ([$sys_descr, $template] as $text) {
if (trim($text) === '') {
continue;
}

foreach ($patterns as $vendor => $pattern) {
if (preg_match($pattern, $text)) {
return $vendor;
}
}
}

return '';
}

/**
* Resolve the catalogue vendor of syslog hosts through their Cacti devices.
*
* A syslog host matches a device by description (the name the collector
* rewrites known hosts to) or by hostname. Results are cached for the request.
*
* @param list<string> $hosts The syslog host names.
*
* @return array<string, string> The vendor of each host, '' when unknown.
*/
function syslog_recommendation_host_vendors(array $hosts): array {
static $cache = [];

$hosts = array_values(array_unique(array_filter(array_map('strval', $hosts), static fn ($host) => $host !== '')));
$todo = array_values(array_filter($hosts, static fn ($host) => !array_key_exists($host, $cache)));

foreach (array_chunk($todo, 500) as $batch) {
$found = array_fill_keys($batch, '');
$marks = implode(',', array_fill(0, count($batch), '?'));

foreach (['description', 'hostname'] as $column) {
$rows = db_fetch_assoc_prepared("SELECT h.$column AS name, h.snmp_sysDescr, ht.name AS template
FROM host AS h
LEFT JOIN host_template AS ht ON ht.id = h.host_template_id
WHERE h.$column IN ($marks)
ORDER BY h.id",
$batch, false);
Comment on lines +580 to +588

foreach (is_array($rows) ? $rows : [] as $row) {
$name = (string) $row['name'];

if (isset($found[$name]) && $found[$name] === '') {
$found[$name] = syslog_recommendation_vendor_from_device((string) ($row['snmp_sysDescr'] ?? ''), (string) ($row['template'] ?? ''));
}
Comment on lines +593 to +595
}
}

$cache = array_merge($cache, $found);
}

$out = [];

foreach ($hosts as $host) {
$out[$host] = $cache[$host] ?? '';
}

return $out;
}

/**
* Find the recommendations that apply to a syslog message.
*
* When the sending device's vendor is known, only that vendor's records and
* vendor-neutral records (an empty vendor) are considered, so a code such as
* an HPE event number does not match the same number from another vendor.
*
* @param string $message The syslog message text.
* @param int $limit The maximum number of recommendations to return.
* @param string $host The syslog host that sent the message, if known.
*
* @return list<array<string, mixed>> The matching rows, longest identifier first.
*/
function syslog_recommendations_lookup(string $message, int $limit = 3): array {
function syslog_recommendations_lookup(string $message, int $limit = 3, string $host = ''): array {
$candidates = syslog_recommendation_candidates($message);

if (!$candidates) {
return [];
}

$vendor = $host === '' ? '' : (syslog_recommendation_host_vendors([$host])[$host] ?? '');
$params = $candidates;
$scope = '';

if ($vendor !== '') {
$scope = " AND vendor IN (?, '')";
$params[] = $vendor;
}

$rows = syslog_db_fetch_assoc_prepared(
'SELECT vendor, identifier, component, severity, message, explanation, recommended_action, reference
FROM syslog_recommendations
WHERE identifier IN (' . implode(',', array_fill(0, count($candidates), '?')) . ')
WHERE identifier IN (' . implode(',', array_fill(0, count($candidates), '?')) . ')' . $scope . '
ORDER BY CHAR_LENGTH(identifier) DESC, vendor
LIMIT ' . max(1, $limit),
$candidates,
$params,
false
);

Expand All @@ -551,42 +653,54 @@ function syslog_recommendations_lookup(string $message, int $limit = 3): array {
/**
* Check the displayed messages for recommendations using bounded, shared queries.
*
* Messages from a host whose vendor is known are only matched against that
* vendor's records and vendor-neutral records, as in syslog_recommendations_lookup().
*
* @param array<int|string, string> $messages The messages keyed by their display row.
* @param array<int|string, string> $hosts The syslog host of each message, keyed like $messages.
*
* @return array<int|string, bool> Whether each message has a matching recommendation.
*/
function syslog_recommendations_for_messages(array $messages): array {
function syslog_recommendations_for_messages(array $messages, array $hosts = []): array {
$available = array_fill_keys(array_keys($messages), false);
$vendors = $hosts ? syslog_recommendation_host_vendors(array_values($hosts)) : [];
$candidates = [];

foreach ($messages as $key => $message) {
$vendor = $vendors[(string) ($hosts[$key] ?? '')] ?? '';

foreach (syslog_recommendation_candidates($message) as $candidate) {
$candidates[$candidate][$key] = true;
$candidates[$vendor][$candidate][$key] = true;
}
}

foreach (array_chunk(array_map('strval', array_keys($candidates)), 500) as $batch) {
// Return candidate spellings so the flags follow the catalogue's SQL collation.
$matches = syslog_db_fetch_assoc_prepared(
'SELECT candidates.identifier
FROM (SELECT ? AS identifier' . str_repeat(' UNION ALL SELECT ?', cacti_sizeof($batch) - 1) . ') AS candidates
WHERE EXISTS (
SELECT 1
FROM syslog_recommendations AS recommendations
WHERE recommendations.identifier = candidates.identifier
)',
$batch
);

if ($matches === false) {
raise_message('syslog_recommendations', __('Recommendations could not be loaded.', 'syslog'), MESSAGE_LEVEL_ERROR);

return array_fill_keys(array_keys($messages), false);
}
foreach ($candidates as $vendor => $vendor_candidates) {
$vendor = (string) $vendor;

foreach (array_chunk(array_map('strval', array_keys($vendor_candidates)), 500) as $batch) {
// Return candidate spellings so the flags follow the catalogue's SQL collation.
$matches = syslog_db_fetch_assoc_prepared(
'SELECT candidates.identifier
FROM (SELECT ? AS identifier' . str_repeat(' UNION ALL SELECT ?', cacti_sizeof($batch) - 1) . ') AS candidates
WHERE EXISTS (
SELECT 1
FROM syslog_recommendations AS recommendations
WHERE recommendations.identifier = candidates.identifier' .
($vendor === '' ? '' : " AND recommendations.vendor IN (?, '')") . '
)',
$vendor === '' ? $batch : array_merge($batch, [$vendor])
);

if ($matches === false) {
raise_message('syslog_recommendations', __('Recommendations could not be loaded.', 'syslog'), MESSAGE_LEVEL_ERROR);

return array_fill_keys(array_keys($messages), false);
}

foreach ($matches as $match) {
foreach ($candidates[$match['identifier']] as $key => $_) {
$available[$key] = true;
foreach ($matches as $match) {
foreach ($vendor_candidates[$match['identifier']] ?? [] as $key => $_) {
$available[$key] = true;
}
}
}
}
Expand Down Expand Up @@ -661,12 +775,13 @@ function syslog_recommendations_labels(): array {
* Build the JSON response the details flyout requests for a row without a record id.
*
* @param string $message The syslog message text.
* @param string $host The syslog host that sent the message, if known.
*
* @return string The JSON document. It always carries the localized labels so
* the browser needs no translation catalogue of its own.
*/
function syslog_recommendations_lookup_json(string $message): string {
$matches = syslog_recommendations_format(syslog_recommendations_lookup($message));
function syslog_recommendations_lookup_json(string $message, string $host = ''): string {
$matches = syslog_recommendations_format(syslog_recommendations_lookup($message, 3, $host));

return (string) json_encode([
'found' => $matches !== [],
Expand Down
2 changes: 1 addition & 1 deletion js/functions.js
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,7 @@ function initSyslogWorkspace() {
note(recSection.dataset.loading);
var call = data.source && data.id
? $.ajax({url: 'syslog.php', type: 'GET', dataType: 'json', data: {action: 'message_details', source: data.source, id: data.id, logtime: data.received || ''}})
: $.ajax({url: 'syslog.php', type: 'POST', dataType: 'json', data: {action: 'recommendation', message: data.message || '', __csrf_magic: csrfMagicToken}});
: $.ajax({url: 'syslog.php', type: 'POST', dataType: 'json', data: {action: 'recommendation', message: data.message || '', host: data.device || '', __csrf_magic: csrfMagicToken}});
call.done(function(result) {
if (token !== request) return;
if (result && result.record) render($.extend({}, result.record, {rules: result.rules || {}}));
Expand Down
2 changes: 1 addition & 1 deletion locales/po/cacti.pot
Original file line number Diff line number Diff line change
Expand Up @@ -3621,7 +3621,7 @@ msgid "A leading <b>%</b> is literal, not a wildcard, and must match the message
msgstr ""

#: syslog_recommendations.php
msgid "All vendors are checked. When several recommendations match, the longest identifiers are shown first, up to three."
msgid "When the syslog host is a Cacti device whose vendor is recognised from its SNMP system description or device template, only that vendor's recommendations and those without a vendor are checked; otherwise all vendors are checked. When several recommendations match, the longest identifiers are shown first, up to three."
msgstr ""

#: syslog_recommendations.php
Expand Down
66 changes: 37 additions & 29 deletions syslog.php
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@

if (get_request_var('action') == 'recommendation') {
header('Content-Type: application/json; charset=UTF-8');
print syslog_recommendations_lookup_json((string) get_nfilter_request_var('message'));
print syslog_recommendations_lookup_json((string) get_nfilter_request_var('message'), (string) get_nfilter_request_var('host'));
exit;
}

Expand Down Expand Up @@ -2737,10 +2737,42 @@ function syslog_messages(string $tab = 'syslog'): void {
$sql_where");
}

$recommendations = syslog_recommendations_for_messages(array_map(
static fn (array $row): string => (string) ($row[$tab === 'syslog' ? 'message' : 'logmsg'] ?? ''),
$syslog_messages
));
// Only the host names for the rows actually shown; never fetch the
// entire syslog_hosts table into memory on every render.
$hosts = [];

if ($tab === 'syslog') {
$host_ids = [];

foreach ($syslog_messages as $sm) {
$host_ids[$sm['host_id']] = true;
}

unset($host_ids['']);

if (cacti_sizeof($host_ids)) {
$bind = array_keys($host_ids);

$hosts = array_rekey(
syslog_db_fetch_assoc_prepared("SELECT host_id, host
FROM `$syslogdb_default`.`syslog_hosts`
WHERE host_id IN (" . implode(', ', array_fill(0, cacti_sizeof($bind), '?')) . ')',
$bind),
'host_id', 'host'
);
}
}

$recommendations = syslog_recommendations_for_messages(
array_map(
static fn (array $row): string => (string) ($row[$tab === 'syslog' ? 'message' : 'logmsg'] ?? ''),
$syslog_messages
),
array_map(
static fn (array $row): string => (string) ($tab === 'syslog' ? ($hosts[$row['host_id'] ?? ''] ?? '') : ($row['host'] ?? '')),
$syslog_messages
)
);
$recommendation_header = [
'display' => '<i class="fa fa-lightbulb syslogRecommendationHeading" role="img" aria-label="' . __esc('Recommendations', 'syslog') . '"></i>',
'align' => 'center',
Expand Down Expand Up @@ -2774,30 +2806,6 @@ function syslog_messages(string $tab = 'syslog'): void {

html_header_sort($display_text, get_request_var('sort_column'), get_request_var('sort_direction'));

// Only the host names for the rows actually shown; never fetch the
// entire syslog_hosts table into memory on every render.
$host_ids = [];

foreach ($syslog_messages as $sm) {
$host_ids[$sm['host_id']] = true;
}

unset($host_ids['']);

if (cacti_sizeof($host_ids)) {
$bind = array_keys($host_ids);

$hosts = array_rekey(
syslog_db_fetch_assoc_prepared("SELECT host_id, host
FROM `$syslogdb_default`.`syslog_hosts`
WHERE host_id IN (" . implode(', ', array_fill(0, cacti_sizeof($bind), '?')) . ')',
$bind),
'host_id', 'host'
);
} else {
$hosts = [];
}

$facilities = array_rekey(
syslog_db_fetch_assoc("SELECT facility_id, facility
FROM `$syslogdb_default`.`syslog_facilities`"),
Expand Down
Loading