diff --git a/.gitignore b/.gitignore index cf7ad1c..6ffdb0c 100644 --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,12 @@ /target map_*.png +track_*.wav *.pdf *.png .DS_Store *.swp +!assets/*.png + +# GUI settings of the last run +sound2view-gui.toml diff --git a/CLAUDE.md b/CLAUDE.md index cbd5ecd..799c78a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -69,35 +69,41 @@ Crate and repo name: `sound2view` (was `audio2video-rs`; local folder may still Workspace (root `Cargo.toml`, resolver 3, `default-members = ["crates/app"]`, so `cargo run --release -- mic` works from the root; `cargo test --workspace` for all tests): - `crates/core` (package `sound2view-core`, lib `sound2view_core`): everything below except `main.rs`. No GUI dependencies. -- `crates/app` (package `sound2view-app`, binary `sound2view`): `main.rs` (CLI), `config.rs`, `gui.rs`, `theme.rs`. +- `crates/app` (package `sound2view-app`, binary `sound2view`): `main.rs` (CLI), `config.rs`, `gui.rs`, `gui_state.rs`, `speech.rs`, `theme.rs`. Depends on the `voice-activity-detection` crate (public git repo, pinned `rev`; its package also builds burn and eframe 0.36, which is why speech detection lives in the app, not in core). Module layout of `crates/core/src` (`foo.rs` + `foo/` style, no `mod.rs`); `main.rs` lives in `crates/app/src`: ``` src/ - lib.rs, peaks.rs, pipeline.rs, render.rs, test_util.rs + lib.rs, peaks.rs, pipeline.rs, render.rs, test_util.rs, tracking.rs geometry/ camera, mic_array, scan_grid source/ capture, simulator/ (scene, signal, reference), video/ (v4l on Linux, avfoundation on macOS) - dsp/ stft, covariance, beamformer/ (delay_and_sum, functional, clean_sc) + dsp/ stft, covariance, beam_signal, beamformer/ (delay_and_sum, functional, clean_sc) ``` +Logo: `assets/` (`logo.svg`, `logo-small.svg` for ≤ 32 px, `logo-wordmark.svg`, PNGs rendered with resvg; `logo-256.png` is the GUI window icon via `include_bytes!` in `main.rs`, see `assets/README.md`). + Dependencies point one way: `geometry` ← `dsp` ← `pipeline` → `render`; `source` only needs `geometry`. -- `main.rs`: CLI. `sound2view [scene.toml]` (default `config.toml` = simulator scene), `mic` (UMA-16 live, `MIC_MAPS` 20 PNGs), `probe`, `camprobe`, `camshot`, `gui`. Loads `sound2view.toml` (`config::CONFIG_PATH`) for all modes except the probes. `type BoxError = Box`. - - `run_configured(&mut source, sample_rate, &config, on_map) -> Result<(), PipelineError>`: builds the `Pipeline` with `config.pipeline`, `config.camera.camera()` and the configured beamformer (`match` on `Method`: `Das::new(.., true, TrueLocation)`, `Functional::new(.., nu, functional_noise_removal, functional_equal_bins)` or `CleanSc::new(.., clean_gain, clean_max_iter, clean_stop_db, clean_beam_cells, true)`), then `run`. One `run` instantiation per method. Used by CLI and GUI; the CLI also uses the configured method (default functional). Core errors convert into `main`'s `Box` with plain `?`. -- `config.rs` (app): `Config { audio, camera, pipeline: PipelineConfig (core), beamformer, display }`, serde with `#[serde(default, deny_unknown_fields)]` everywhere: file, sections and entries optional, typos are errors. `Config::load(path)`: missing file → defaults (prints a note). `AudioConfig { device, sample_rate }`, `CameraConfig { name, width, height, fps, hfov_deg }` + `camera()` (pinhole via `from_hfov`, default `Mount`), `Method` (`das`/`functional`, `ALL`, `label`), `BeamformerConfig { method, nu }`, `DisplayConfig { db_range }`. Example with all defaults: `sound2view.toml` in the root. +- `main.rs`: CLI. `sound2view [scene.toml]` (default `config.toml` = simulator scene), `mic [maps]` (UMA-16 live, `MIC_MAPS` 20 PNGs unless given), `probe`, `camprobe`, `camshot`, `gui`. Loads `sound2view.toml` (`config::CONFIG_PATH`) for all modes except the probes. `type BoxError = Box`. `run_gui` limits rayon's global pool to all cores but `GUI_FREE_CORES` 2 (`leave_cores_for_gui`; with all cores computing maps the camera image stuttered on the Mac mini) and applies the saved GUI settings. + - `run_configured(&mut source, sample_rate, &config, on_map) -> Result<(), PipelineError>`: builds the `Pipeline` with `config.pipeline`, `config.camera.camera()` and the configured beamformer (`match` on `Method`: `Das::new(.., true, TrueLocation)`, `Functional::new(.., nu, functional_noise_removal, functional_equal_bins)` or `CleanSc::new(.., clean_gain, clean_max_iter, clean_stop_db, clean_beam_cells, true)`), then `run`. One `run` instantiation per method. Used by CLI and GUI; the CLI also uses the configured method (default CLEAN-SC). Core errors convert into `main`'s `Box` with plain `?`. +- `config.rs` (app): `Config { audio, camera, pipeline: PipelineConfig (core), beamformer, display, peaks, tracking, speech }`, serde with `#[serde(default, deny_unknown_fields)]` everywhere: file, sections and entries optional, typos are errors. `Config::load(path)`: missing file → defaults (prints a note). `AudioConfig { device, sample_rate }`, `CameraConfig { name, width, height, fps, hfov_deg }` + `camera()` (pinhole via `from_hfov`, default `Mount`), `Method` (`das`/`functional`, `ALL`, `label`), `BeamformerConfig { method, nu }`, `DisplayConfig { db_range }`. Example with all defaults: `sound2view.toml` in the root. - `gui.rs` (eframe 0.37: `App::ui(&mut self, ui, frame)`, panels via `egui::Panel::top(..).show(ui, ..)` and `CentralPanel::default().show(ui, ..)`; `show_inside` is deprecated). `sound2view gui`, window starts maximized (`ViewportBuilder::with_maximized`). `App::new(cc, config)`. - - Camera thread (`spawn_camera`): opens the camera selected in the GUI (`CameraChoice::Auto` = `CameraSource::open_by_name(camera.name)`, or `CameraChoice::Index(i)` = `CameraSource::new(i)`) inside the thread, `sync_channel(1)` + `try_send` (drops frames), `request_repaint` per frame. Refreshes `list_cameras()` on every open attempt (also while missing, so plugged-in cameras show up). Reopens immediately when the selection changes (old camera dropped first), retries every 2 s (`RETRY`; `wait_for_other_choice` polls the selection every 100 ms meanwhile), reopens after 10 consecutive frame errors (`CAMERA_MAX_FAILED_FRAMES`). - - `CameraShared { choice: Mutex, status: Mutex, cameras: Mutex> }` in an `Arc`, shared by GUI and camera thread. `camera_control()` in the top bar: "Camera" `ComboBox` with "Auto ()" plus every listed camera as "name (index)"; a change sets the choice and status `Opening`. Unfiltered: on Linux the icspring metadata node is listed too (selecting it shows the open error). `hfov_deg` belongs to the configured camera; other cameras only fit the overlay with the same field of view. + - Camera thread (`spawn_camera`): opens the camera selected in the GUI (`CameraChoice::Auto` = `open_by_name(camera.name)`, or `CameraChoice::Name(name)` = `open_by_name(name)`; by name, since the index changes on replugging) inside the thread, `sync_channel(1)` + `try_send` (drops frames), `request_repaint` per frame. Refreshes `list_cameras()` on every open attempt (also while missing, so plugged-in cameras show up). Reopens immediately when the selection changes (old camera dropped first), retries every 2 s (`RETRY`; `wait_for_other_choice` polls the selection every 100 ms meanwhile), reopens after 10 consecutive frame errors (`CAMERA_MAX_FAILED_FRAMES`). Measures the delivered size and frame rate (`FrameRate`, once a second) into `CameraShared::measured` ("1920x1080, 30 fps"), shown in the camera status tooltip and the settings menu. + - `CameraShared { choice: Mutex, status: Mutex, cameras: Mutex> }` in an `Arc`, shared by GUI and camera thread. `camera_control()` in the settings menu: "Camera" `ComboBox` with "Auto ()" plus every listed camera name (deduplicated); a change sets the choice and status `Opening`. Unfiltered: on Linux the icspring metadata node is listed too (selecting it shows the open error). `hfov_deg` belongs to the configured camera; other cameras only fit the overlay with the same field of view. - `DeviceStatus` (`Opening`, `Missing(error)`, `Running`, `Lost(error)`) for camera and mic, each in an `Arc>`, set by its thread via `set_status` (with `request_repaint`). Top bar: dot + short text per device, full message as tooltip. Camera message centered over the dimmed last frame (or alone before the first frame); the map overlay is hidden while the mic is not `Running` (stale map). - - Audio thread (`spawn_audio`): outer loop opens the mic (`Missing` + retry every 2 s on failure). Inner loop: read `PipelineSettings { beamformer, pipeline }` from an `Arc>`, put them into its `Config` copy, `run_configured`. `on_map` sends `MapData { size, power }` (raw power), returns `Break` when the settings changed (restart with the mic kept open, ALSA `hw:` is exclusive) or the GUI closed (thread ends). A pipeline error (e.g. mic unplugged: `CaptureError` via the source) drops the mic, sets `Lost(error)` and reopens after 2 s. + - Audio thread (`spawn_audio`): outer loop opens the mic (`Missing` + retry every 2 s on failure). Inner loop: read `PipelineSettings { beamformer, pipeline, min_above_floor_db }` from an `Arc>`, put them into its `Config` copy, `run_configured`. `on_map` runs the speech detector (loaded once per thread, reset per run) and sends `MapData { size, power, tracks, speech }` (raw power, tracks without audio), returns `Break` when the settings changed (restart with the mic kept open, ALSA `hw:` is exclusive) or the GUI closed (thread ends). A pipeline error (e.g. mic unplugged: `CaptureError` via the source) drops the mic, sets `Lost(error)` and reopens after 2 s. - `theme.rs` (app): dark theme applied in `App::new` (`theme::apply`, forces `Theme::Dark`, `all_styles_mut`). Palette constants: `BACKGROUND` #0f1113 (behind the image), `PANEL` #181b1f (top bar), `TEXT`, `TEXT_WEAK`, `ACCENT` amber #f5a524 (matches the overlay), `OK`, `ERROR`; 6 px radius, larger text (body 15) and padding. - - `ui`: top bar (`Frame` with `PANEL`): title, `controls()`: beamformer `ComboBox`, the selected method's parameter (functional: nu slider 1–100, log; CLEAN-SC: "Sources" = `clean_max_iter` 1–30; DAS: none), band sliders f_lo/f_hi (100–6000 Hz, log, each limited by the other, min width 100 Hz), overlay dB range slider (1–20 dB); mic and camera status right-aligned. Pipeline settings are copied to the shared settings only while no pointer button is down (dragging does not restart the pipeline on every step). The overlay RGBA is computed in the GUI (`render::map_rgba` with the current dB range) from the last `MapData`, so the dB range applies immediately, also between maps. Central panel (`BACKGROUND`, 12 px margin) paints camera and map texture (`TextureOptions::LINEAR`) with `painter.image` on the same rect: largest rect with the image aspect ratio, centered in the panel. GUI changes are not written back to the file. -- `pipeline.rs`: `PipelineConfig { fft_len 1024, memory_frames 40, map_every 10, f_lo_hz 900, f_hi_hz 4100, grid_width 64, grid_z 2.0 }` (serde, `Default` = these values; memory 40 ≈ 0.43 s, one map per 10 frames ≈ 0.11 s ≈ 9 maps/s, `power_map` ≈ 85 ms must fit). - - `Pipeline> { stream, cov, beamformer, bins, map_every, layout }` (private fields). `Pipeline::<16, B>::new(sample_rate, &PipelineConfig, camera: Camera, make_beamformer: impl FnOnce(Steering<16>) -> B)` builds scan grid, bins (upper end clipped to the last bin) and `Steering`, then the beamformer from it. The closure guarantees grid and beamformer match. Asserts f_lo < f_hi and map_every > 0 (TODO: `Result` with thiserror). - - `run(&mut source, pipeline, on_map)`: read → `StftStream::push` → `SlidingCovariance::accumulate`; every `map_every` frames `power_map` on the current estimate (timed, printed) → `on_map(&power, &layout) -> Result, Box>`. Stops at `Ok(0)`, on `Break`, or on the first error. CLI: `save_maps(src_pos, max_maps)` in `main.rs` writes PNGs; GUI: `spawn_audio` in `gui.rs` sends raw power maps. Takes `&mut S` so `run_mic` can print `dropped()` afterwards. -- `peaks.rs`: `PeakConfig { enabled, range_db 6, min_dist_cells 4, max_count 5 }` (serde, `[peaks]` in `sound2view.toml`), `Peak { col, row (grid cells, sub-cell), level_db }`, `find_peaks(power, [w, h], config)`: local maxima (8 neighbors, plateau gives its first cell), within `range_db` of the maximum, greedy minimum distance (stronger kept), at most `max_count`, strongest first; per axis a parabola through the log power (exact for a Gaussian), none on the border. GUI: blue dot (`theme::PEAK`) with dB label, "Peaks" checkbox; CLI PNGs: blue dot. Also the tool for the `Mount` alignment (sub-cell peak positions). Tests: Gaussian at (10.3, 7.6) exact, two peaks with levels, range/distance/count, plateau and empty map. -- `render.rs`: `map_image(power, layout, src_pos, peaks) -> RgbImage`, independent of `Pipeline` (reusable for the GUI). Grid image → `imageops::resize` (Nearest) to camera size → red cross per true source (`camera.project`, skipped if z ≤ 0 or outside). `power_to_rgb`: gray, 0 dB = white, −15 dB and below = black (hard-coded `15.0`; to become a `DB_RANGE` constant). Sim and mic files overwrite each other. + - `ui` (redesigned after the Mac mini demo, where the one-row top bar collided at 1920 px): no top bar; the image fills the window. `controls_overlay`: an `egui::Area` in the top-right corner (`CONTROLS_OFFSET`) with the mic and camera status (dot + short text, full message and the measured camera format as tooltip) and the "⚙ Settings" `MenuButton`; faint (`CONTROLS_IDLE_OPACITY` 0.3, `ui.set_opacity`) until the pointer is within `CONTROLS_HOVER_MARGIN` of the previous frame's `controls_rect` or the menu is open. `menu_contents`: all settings in sections, sliders `MENU_SLIDER_WIDTH` 160 px: Beamformer (`ComboBox`, nu or "Sources" = `clean_max_iter`, the method's checkboxes), Map (`band_slider` with two handles, overlay dB range 0.1–20 log, "Skip transients"), Sources ("Show sources", "Above floor" 0–15 dB, "Coherence" −20–0 dB = `peaks.min_coherence_db`; display thresholds, apply immediately), Speech (checkbox, threshold 0.3–0.99), Camera (`camera_control`, measured format), a note where changes are saved. Pipeline settings are copied to the shared settings only while no pointer button is down (dragging does not restart the pipeline on every step); the GUI state is saved at the same moment if it changed. The overlay RGBA is computed in the GUI (`render::map_rgba` with the current dB range) from the last `MapData`, so the dB range applies immediately, also between maps. Central panel (`BACKGROUND`, 12 px margin) paints camera and map texture (`TextureOptions::LINEAR`) with `painter.image` on the same rect: largest rect with the image aspect ratio, centered in the panel; `draw_tracks` draws the tracks with `Track::shown` (hover: "+x dB above the noise floor (needs y), z dB coherent above 2 kHz"). +- `gui_state.rs` (app): `GuiState` (all `Option`s: `beamformer`, `f_lo_hz`, `f_hi_hz`, `skip_transients`, `db_range`, `tracks`, `min_above_floor_db`, `min_coherence_db`, `speech`, `speech_threshold`, `camera` name), saved to `sound2view-gui.toml` in the working directory (`GUI_STATE_PATH`, git-ignored). Only the differences from `sound2view.toml` are saved (`changes_from(base)`, `base = of_config(sound2view.toml)`), so later edits of `sound2view.toml` stay in effect for everything not changed in the GUI. `run_gui`: `load()` (missing or unreadable: empty), `apply(&config)` (ignored as a whole if the result fails `Config::validate`), the saved camera name to `App::new`. The GUI saves when the differences change (`save()`, errors printed). Tests: round trip of the differences, invalid values ignored. +- `speech.rs` (app): speech detection per track with voice-activity-detection (`Preprocessor::push(&[f32])` at 16 kHz, 62 features per 10 ms frame, `Model::predict`, an MLP on burn). `SpeechConfig { enabled, model "model/speech_detector.json" (copy of the classifier's model, relative to the working directory), on_threshold 0.95, hold_ms 500, smoothing_ms 50, min_onset_ms 80, echo_correlation 0.2, echo_margin_db 6, max_shift_cells 3, gain_db 36, min_level_db -55, min_source_db -6 }` (`[speech]`, `validate`). `SpeechDetector::load`, `reset()` per pipeline run (ids start over), `update(&tracks)` (one `Preprocessor` and `Hysteresis` per track id, gone ids forgotten), `status(id)`/`statuses()` -> `SpeechStatus { speech, probability, echo_of }`. Per classifier frame: track audio × `gain_db` (the model and the −55 dB gate expect microphone levels, speech ≈ −25 dBFS; the UMA-16 gives ≈ −60 dBFS at 1–2 m), p = 0 below `min_level_db`, while the track is not `steady` (impulsive: snapping) or while `Track::source_upper_db < min_source_db` (the track's share of the upper-band power not explained by other tracks; speech leaking into the beam from a talker elsewhere, also outside the image: below ≈ 1.5 kHz the array has no directivity, so the classifier alone cannot tell whose speech a beam carries; above 2 kHz a talker elsewhere leaks in at the sidelobe level). `Hysteresis` (`HysteresisParams`): smoothed p += w·(p − smoothed), w = 1 − exp(−10 ms/τ); speech starts when the smoothed p has been ≥ `on_threshold` for `min_onset_ms` (snaps and claps reach 0.7 for one 10 ms frame at most, speech for 50 ms to over 1 s) and the frame's own p is too; ends `hold_ms` after the last frame whose own p ≥ `on_threshold` (own p, so the smoothing tail cannot re-arm or restart it; a half speech-like sound after the speech, p ≈ 0.3–0.5, does not keep it), or at once when the track moved more than `max_shift_cells` from the position of that frame (a source nearby that took over the track). Echo check after each update (`echo_source`, on the speech flags before any is cleared): a speech track is an echo of another speech track with ≥ `echo_margin_db` more speech energy (`TrackSpeech::speech_energy` = Σ p·rms² over the last 0.3 s, `ECHO_WINDOW`) whose audio correlates ≥ `echo_correlation` (`max_correlation`, normalized, ±20 ms `ECHO_MAX_LAG`): speech off, `echo_of = Some(id)`, GUI label "echo of #N". Speech energy, not loudness: a snapping track next to the speaker is louder, speech-like through the leaked speech and correlates only ≈ 0.35 (the snaps dilute it), but has 9–18 dB less speech energy; two talkers at similar levels both stay speech. GUI: "Speech" checkbox (from `speech.enabled`, `Arc`; the audio thread loads the model on first use and resets the detector while off) and `on_threshold` slider (0.3 to 0.99, `Arc` as f32 bits, applied per map via `set_on_threshold`, no pipeline restart); CLI: `main::load_speech` (`None` if disabled or the model fails), green dot in the PNG (`map_image(.., speech_ids)`) and a line per speech start/stop. Tests: hysteresis (hold, syllables and pauses, a 0.45 sound after speech, a 4-cell shift ends it, 60 ms bursts do not start it), level gate, `testdata/speech_16k.wav` (macOS `say`, 4.7 s) is speech in 40 of 44 maps, the same with `source_upper_db` −12 or `steady` false never, white noise and a 1 kHz tone never, correlation of a delayed 0.3× copy > 0.9 and of independent noise < 0.1, a reflection (5 ms, 0.3×) never speech while the speaker is, speaker plus louder snapping with mutual leakage (speaker never an echo, snapping never speech), two talkers with 0.3 mutual leakage each speech in ≥ 70 % of the maps they are alone. +- `pipeline.rs`: `PipelineConfig { fft_len 1024, memory_frames 40, map_every 10, f_lo_hz 900, f_hi_hz 4100, grid_width 64, grid_z 2.0, grid_margin_deg 15, skip_transients false }` (serde, `Default` = these values; memory 40 ≈ 0.43 s, one map per 10 frames ≈ 0.11 s ≈ 9 maps/s, `power_map` ≈ 85 ms must fit). **Scan grid margin** (`grid_margin_deg`, validated 0 ≤ x < 90): the grid reaches that far beyond the field of view on every side at the same cell size, capped at `MAX_OFF_AXIS_DEG` 55 off axis (`ScanGrid::with_margin`), so a source outside the image has its maximum on a margin cell instead of the image border (the main lobe, ≈ 34° at 3 kHz, reached in from up to ≈ 17° outside; with the 83° × 53° camera the vertical margin matters most) and CLEAN-SC takes it out of the visible part (DAS gains nothing, the lobe still reaches the border). Beamformer, `Steering` and floor subgrid see the full grid (`Pipeline::grid_size`); `layout`, `on_map` (`Viewport::crop` into a visible buffer), tracks and `TrackAudio` are in visible cells (`Pipeline::point` adds the viewport offset). With the 83° camera and width 64: 104 × 64 cells instead of 64 × 36 (≈ 2.9× map time); 0 turns it off. Test: a 10 dB louder sine 52° above the axis next to one inside: visible maximum and only confirmed track at the inside source, top row < −15 dB with the margin, > −6 dB without. + - `Pipeline> { stream, cov, beamformer, bins, map_every, layout }` (private fields). `Pipeline::<16, B>::new(sample_rate, &PipelineConfig, &PeakConfig, &TrackerConfig, camera: Camera, make_beamformer: impl FnOnce(Steering<16>) -> B)` builds scan grid, bins (upper end clipped to the last bin) and `Steering`, then the beamformer from it. The closure guarantees grid and beamformer match. Asserts f_lo < f_hi and map_every > 0 (TODO: `Result` with thiserror). + - `run(&mut source, pipeline, on_map)`: read → `StftStream::push` → `SlidingCovariance::accumulate` into `cov` and, for frames that are not transients (energy over the map band `TRANSIENT_DB` 10 dB above the median of the last `TRANSIENT_HISTORY` 24 frames, `is_transient`), into `steady_cov`; with `skip_transients` transients stay out of `cov` too (impulsive sounds vanish from the map, which was tried as the default and rejected: the user wants snapping shown, just not green); the beams get every frame. Per map `power_map` on `cov` and (unless skipping, then a copy) on `steady_cov` (`steady`, second map ≈ 11 ms CLEAN-SC); `Track::steady` = a peak of the steady map within `peaks.min_dist_cells`, at most `STEADY_RANGE_DB` 6 dB below its maximum; only steady tracks can be speech. Why: a hand snapping next to a talker is inside the beam toward the talker at every speech frequency (±20° at 3 kHz), so both beam signals are the same audio and no audio test, correlation or energy margin can tell the snapping track from the talker (the snaps turned green); the deconvolved steady map has no spot at the hand. The first ≈ 0.13 s of a sound after silence count as transient too; every `map_every` frames `power_map` on the current estimate (timed, printed) → `update_tracks` (`find_peaks`; the largest coherent power (`coherent_power`, map band, current covariance) over a coarse subgrid (every `FLOOR_STRIDE` 8th cell, ≈ 32 points, ≈ 2 ms) goes into `floor_samples` (linear, the last `FLOOR_WINDOW_S` 60 s of maps, ≤ 0 ignored; with uncorrelated noise it fluctuates around 0, so a single point such as the map maximum is useless in dB, the max of 32 is stable), the noise floor is their minimum (`floor`, minimum statistics: the quietest map of the window sets it, a pause between words is enough; a 10th percentile needed quiet for a tenth of the window and rose onto a continuous talker on hardware; infinite without samples, so nothing passes before the first map, and a source present from the start shows only after a quieter moment); `Tracker::update` with all peaks; each track gets `source_db` (coherent power above the floor) and `source_upper_db` (over `upper_bins`, the map bins from `UPPER_BAND_HZ` on, all if the band ends below: the track's coherent power divided by the received power minus the coherent power of the other tracks, at least `UPPER_UNEXPLAINED_MIN` 0.1 of the received power; as a plain fraction of the received power, snapping next to the talker raised the upper-band power ≈ 10 dB and pushed the talker below `speech.min_source_db`, speech off within the hold; a talker outside the image is no track and still shows as leakage). Power, not fraction, for the floor: a fraction is bounded by 1, and in a real room the quiet state has a steady coherent source (fan) with a fraction near 1, so a talker could never be 8 dB above it (first hardware test: no dots at all); a talker has more coherent power than the fan, each track gets its beam's samples since the previous map, beams of dropped tracks removed, new tracks get a `BeamSignal`, all re-steered to the grid cell at `grid_z` via the scaled camera; `TrackAudio` is `None` without integer decimation) → `on_map(&power, &layout, &tracks) -> Result, Box>`. Every STFT frame also goes to all beams. The pipeline keeps its own `Steering` (clone of the beamformer's). Tests: two sines after 2 s of quiet (`Chain` of two simulators) with CLEAN-SC give two confirmed tracks within 0.34 cells, each > 10 dB above the floor and > −5 dB coherent above 2 kHz, own source 30/12 dB above the other in the track audio (DAS pulls the two peaks about 2 cells together); 6 s of 80 dB mic noise alone give no track above `min_above_floor_db` (the closest came to +4.2 dB); a steady source from the start is the floor and stays below it, a source 14 dB louder appearing later is above it, at the right place; a talker next to a 10 dB louder source in the upper band keeps an upper-band share > −4 dB; snapping (`Snapping` source: band noise in 8 ms bursts every 300 ms) next to a talker gives one shown steady track at the talker with `skip_transients` and, without, a second one at the hand that is not steady. Stops at `Ok(0)`, on `Break`, or on the first error. CLI: `save_maps(src_pos, max_maps, show_tracks)` in `main.rs` writes PNGs and, via `TrackRecorder` (hound), `track_NNN.wav` per confirmed track (16 kHz mono f32, from confirmation on); GUI: `spawn_audio` in `gui.rs` sends raw power maps. Takes `&mut S` so `run_mic` can print `dropped()` afterwards. +- `peaks.rs`: `find_peaks_in(power, full_size, viewport, config)`: peaks inside the `Viewport` only, positions, `range_db` and `center_weight_db` relative to it; maxima outside take part in the `min_dist_cells` check (an outside source suppresses the maximum its lobe leaves at the border) but not in `max_count`; `edge_margin_cells` at the full-map border. `find_peaks` = full viewport. Test: outside Gaussian hides the border maximum, inside peak comes out shifted. `PeakConfig { enabled, range_db 12 (was 6: a talker next to snapping is ≈ 10 dB below the map maximum and lost its peak; "nothing" is decided against the floor now), min_dist_cells 4, max_count 5, min_above_floor_db 8 }` (serde, `[peaks]` in `sound2view.toml`), `Peak { col, row (grid cells, sub-cell), level_db }`, `find_peaks(power, [w, h], config)`: local maxima (8 neighbors, plateau gives its first cell), within `range_db` of the maximum, greedy minimum distance (stronger kept), at most `max_count`, strongest first; per axis a parabola through the log power (exact for a Gaussian), none on the border. `min_above_floor_db` is applied by the app when showing and recording tracks (`Track::source_db`, measured by the pipeline, which needs the covariance): a track is shown only if its coherent power over the map band is that far above the noise floor; without it every map has a maximum and silence shows a dot after `min_hits` maps. Computed in the pipeline; drawn as tracks (see `tracking.rs`): GUI (`draw_tracks`, sizes relative to the drawn image height with minimums): hollow blue ring (`theme::PEAK`) on a dark fill, speech: larger filled green dot (`theme::SPEECH`) with a halo; label "#id level dB probability" on a pill (dark, or green with dark text and "speech"), "Tracks" checkbox (`peaks.enabled`); CLI PNGs blue dot. Also the tool for the `Mount` alignment (sub-cell peak positions). Tests: Gaussian at (10.3, 7.6) exact, two peaks with levels, range/distance/count, plateau and empty map. After the Mac mini demo (dozens of dots on noise and along the image border): `edge_margin_cells` 1 at the time (peaks whose refined position is closer to the border are dropped before the distance check, so they do not suppress a peak inside: a source outside the field of view has its maximum on the border, since the scan grid ends there; now 0, since with the scan grid margin the grid border lies far outside the image and a maximum there should suppress its leakage; set 1 with `grid_margin_deg` 0), `center_weight_db` 6 at the time (now 2, the margin handles outside sources) (`center_penalty_db(col, row, size, config)` = weight · squared distance from the map center in half map widths: 0 in the center, the full weight at the left/right border, about a third at top/bottom of 16:9; ranks candidates for `max_count` and raises the shown threshold, `Track::edge_penalty_db`), `min_coherence_db` −8 (`Track::source_upper_db` must reach it: a level-independent test, point sources and snaps ≈ 0 dB, diffuse room noise and mic noise far below; GUI slider "Coherence"). `Track::shown(PeakConfig)` = confirmed, `source_db − edge_penalty_db ≥ min_above_floor_db` and `source_upper_db ≥ min_coherence_db`; used by GUI and CLI. Tests: border maximum dropped without suppressing a peak inside, central peak preferred, penalty 0/full weight. +- `tracking.rs`: `TrackerConfig { max_dist_cells 4 (was 6: noise maxima wander and kept continuing one track), min_hits 3, confirm_window 5 (confirmed after a matching peak in min_hits of the last confirm_window maps, kept as bits in `Track::recent`; noise maxima do not stay long enough, a snap stays about 4 maps in the covariance and is confirmed), max_missed 5, smoothing 1.0 (0.5 put the dot between two spots when a CLEAN-SC maximum jumped to a reflection: "CLEAN-SC off, functional on point" on hardware; on `julian/claude` the dots were the raw peaks) }` (serde, `[tracking]`, `validate`), `Track { id, col, row (smoothed, grid cells), level_db, source_db (coherent power over the map band in dB above the noise floor), source_upper_db (coherent fraction from `UPPER_BAND_HZ` 2 kHz on, absolute), both filled by the pipeline, hits, missed, confirmed, samples: Vec }`, `Tracker::update(&peaks)`: greedy nearest (track, peak) pairs within the gate, position += smoothing·(peak − position), unmatched peaks start tracks (ids never reused), unmatched tracks count a miss and are dropped after more than `max_missed`; `confirmed` after `min_hits` matching maps in total (not in a row: hits are not reset by a miss, since a source at the source gate's limit flickers and never got three in a row on hardware), sticky. Only confirmed tracks are drawn (dot persistence against flicker). Tests: moving peak keeps id, jump starts a track, confirmation with misses in between/drop, nearest pairs first. +- `dsp/beam_signal.rs`: audio from one direction for the speech classifier (`/Users/onmcu/Git/voice-activity-detection`, `Preprocessor::push(&[f32])` at 16 kHz, next step). `TRACK_SAMPLE_RATE` 16 kHz, `decimation_factor(f_s)` (integer only, 44.1 kHz gives none). `BeamSignal::new(fft_len, stft_scale, f_s, distances)`, `steer(distances)`: w_k = a·e^{jωτ_ref}/(aᴴa) per bin (all N/2 + 1), τ_ref = mean mic distance / c, `push_frame(snapshots)`: y = wᴴx, real DC/Nyquist, inverse FFT, overlap-add with gain 1/(N·scale) (periodic Hann sums to 1 at hop N/2; the weights shift each mic's frame circularly only by (r_m − r_ref)/c ≤ about 13 samples, where the window is ≈ 0. Removing the whole delay r_m/c, ≈ 280 samples at 2 m, wrapped the window around the frame: error at every hop, heard as a 94 Hz buzz, fixed), decimated by a Blackman windowed-sinc FIR (32·factor − 1 taps, cutoff 0.4·16 kHz), `take_output()`. A source at the point comes out as s(t − τ_ref) at 1 m. Tests: band noise at off-bin frequencies reconstructed to 1e-7 (factor 1; a bin-centered sine is N-periodic and hides a wrong circular shift, 2e-2 with the old weights), decimated amplitude, about 17 dB suppression of a source 62° away at 3.4 kHz, decimator pass/stopband with uneven chunks. +- `render.rs`: `map_image(power, layout, src_pos, tracks, speech_ids) -> RgbImage` (dot per confirmed track, green if its id is in `speech_ids`), independent of `Pipeline` (reusable for the GUI). Grid image → `imageops::resize` (Nearest) to camera size → red cross per true source (`camera.project`, skipped if z ≤ 0 or outside). `power_to_rgb`: gray, 0 dB = white, −15 dB and below = black (hard-coded `15.0`; to become a `DB_RANGE` constant). Sim and mic files overwrite each other. - `source/capture.rs` (cpal 0.18.2; `device.id()`/`description()`, not `name()`): - `probe()`: per host, matching input devices (`UMA16_PATTERNS` "uma"/"minidsp") with their configs; devices with > 20 configs are plug devices, skipped. Result on Linux: `alsa:hw:CARD=UMA16v2,DEV=0`, 16 ch, I32, fixed rates 8000/11025/12000/16000/32000/44100/48000. Use the card name, not the index. - `CpalSource { _stream, rx, pending: VecDeque, dropped: Arc, stream_error: Arc>>, sample_rate }`. `new(pattern, sample_rate) -> Result`: first input device whose id or name contains the pattern, `StreamConfig` with M channels, default buffer. Callback converts samples to f64 (see below) and `try_send`s on a `sync_channel(64)`; when full the chunk is dropped and counted (audio thread never blocks). Error callback: `Xrun`, `RealtimeDenied`, `DeviceChanged` are printed (stream keeps running); any other `cpal::ErrorKind` (e.g. `DeviceNotAvailable` on unplug) is stored as the first fatal error. `dropped()` getter. @@ -109,7 +115,7 @@ Dependencies point one way: `geometry` ← `dsp` ← `pipeline` → `render`; `s - `source/video.rs` (+ `video/v4l.rs`, `video/avfoundation.rs`): camera capture behind the same API on both platforms, backend chosen with `#[cfg(target_os)]` (`pub use` of `CameraSource`, `list_cameras`, `initialize`, `is_authorized`, `probe`). Shared: `VideoError` (`NotFound(name)`, `NoSuchIndex`, `NotAuthorized`, `Camera` (Linux: `NokhwaError`, macOS: `String` from `NSError`), `NoFrame(Duration)`, `Closed`), `CameraEntry { index, name }`, `CameraSource::open_by_name` (case-insensitive substring, tries every match in order). - `v4l.rs` (Linux): nokhwa 0.10 with V4L2. `new(index = /dev/videoN, w, h, fps)`: MJPEG `Exact`, else `Closest`; RGBA via nokhwa's decoder. `initialize` no-op, `is_authorized` always true (`video` group, see README). `probe` opens each camera to list its formats. - - `avfoundation.rs` (macOS): own backend on objc2 (nokhwa's AVFoundation bindings are broken, see the macOS section). `devices()` = `AVCaptureDeviceDiscoverySession` with `AVCaptureDeviceTypeExternal` (macOS 14+) and `BuiltInWideAngleCamera`; index = position in that list, name = `localizedName`. `new`: pick the (format, frame rate range) pair closest to the request (resolution mismatch in pixels first, then fps outside the range; `FormatOption::distance`), so format and frame rate always come from the same entry; `AVCaptureSession` + `AVCaptureDeviceInput` + `AVCaptureVideoDataOutput` with `videoSettings` = 32BGRA (AVFoundation converts MJPEG/YUV), `setActiveFormat` and min = max frame duration (`frame_duration`: the range's own CMTime at its ends, else 1/fps) under `lockForConfiguration` inside begin/commitConfiguration, then `startRunning`. Delegate `FrameDelegate` (`define_class!`, ivars = `SyncSender`) on a serial dispatch queue: `to_rgba` locks the CVPixelBuffer, copies BGRA → RGBA row by row (stride), `try_send` (drops while the reader is behind). `next_frame` = `recv_timeout(2 s)`. `Drop` stops the session. `initialize` = `requestAccessForMediaType`, `is_authorized` = `authorizationStatusForMediaType == Authorized`. `probe` lists devices with every format and fps range (`1920x1080 @ 5-30 FPS, 420v`) without opening a session. Everything that returns autoreleased objects runs in `autoreleasepool`. + - `avfoundation.rs` (macOS): own backend on objc2 (nokhwa's AVFoundation bindings are broken, see the macOS section). `devices()` = `AVCaptureDeviceDiscoverySession` with `AVCaptureDeviceTypeExternal` (macOS 14+) and `BuiltInWideAngleCamera`; index = position in that list, name = `localizedName`. `new`: pick the (format, frame rate range) pair closest to the request (resolution mismatch in pixels first, then fps outside the range; `FormatOption::distance`), so format and frame rate always come from the same entry; `AVCaptureSession` + `AVCaptureDeviceInput` + `AVCaptureVideoDataOutput` with `videoSettings` = 32BGRA (AVFoundation converts MJPEG/YUV), `setActiveFormat` and min = max frame duration (`frame_duration`: the range's own CMTime at its ends, else 1/fps) under `lockForConfiguration` inside begin/commitConfiguration, then `startRunning`. Delegate `FrameDelegate` (`define_class!`, ivars = `SyncSender`) on a serial dispatch queue: `to_rgba` locks the CVPixelBuffer, copies BGRA → RGBA row by row (stride), `try_send` (drops while the reader is behind). `next_frame` = `recv_timeout(2 s)`. `Drop` stops the session. `initialize` = `requestAccessForMediaType`, `is_authorized` = `authorizationStatusForMediaType == Authorized`. `probe` lists devices with every format and fps range (`1920x1080 @ 5-30 FPS, 420v`) without opening a session. Everything that returns autoreleased objects runs in `autoreleasepool`. After the demo (image far below full HD and laggy on the Mac mini): `FormatOption::distance` = (too slow: max fps below min(requested, `MIN_LIVE_FPS` 25), resolution mismatch, fps outside the range), so an uncompressed 1080p at 5 fps no longer wins; `videoSettings` also sets `kCVPixelBufferWidthKey`/`HeightKey` to the format's size (`cf_key` bridges the CFString keys); the input-priority preset is set implicitly by `setActiveFormat` (an explicit `setSessionPreset(AVCaptureSessionPresetInputPriority)` throws an Objective-C exception, which aborts the process, seen on the Mac mini); the chosen format is always printed ("camera 0: 1920x1080 at 5-30 fps, 420v (requested ...)"); `to_rgba` writes into a preallocated buffer. Type-checked with `cargo check --target aarch64-apple-darwin` on Linux (target installed), not yet run on the Mac. - `lib.rs`: `Vector3 = nalgebra::Vector3`, `Point3 = nalgebra::Point3`, `Complex64 = nalgebra::Complex` (same type as `realfft::num_complex::Complex`, single num-complex 0.4 in the lock file), `C_SOUND = 343.0` (TODO: rename to `SPEED_OF_SOUND`), `distance_to_delay(d) = d / C_SOUND`, the traits above. - `geometry/mic_array.rs`: `pub struct MicArray { pub mic_pos: [Point3; M] }`, indexed by USB channel. - `uniform_rect(cols, rows, pitch_x, pitch_y, channel_of: impl Fn(row, col) -> usize)`: asserts `rows * cols == M` and each channel assigned once. Row 0 = top = +y, col 0 = left = −x, viewed from the sound-source side (z into the scene). @@ -124,22 +130,23 @@ Dependencies point one way: `geometry` ← `dsp` ← `pipeline` → `render`; `s - `Mount { pub offset: Vector3 (m, array frame), pub rotation: UnitQuaternion }`; `Default` / `from_offset` use 180° about z. `pose() -> Isometry3`: p_array = R·p_cam + t. - `Camera::unproject(u, v, z)` = pose ∘ intrinsics; `project(&Point3)` = intrinsics ∘ pose⁻¹. z is the depth from the camera, not the array z. - Tests: round trip, image center on axis, top-left pixel is +x/+y (pins the rotation), offset mount (round trip with depth z − offset.z, center maps to offset + z·ẑ). -- `geometry/scan_grid.rs`: `ScanGrid { width, height: usize, points: Vec }` (private, getters). `new(camera, width, z)` unprojects every pixel center of the `scaled` camera at depth z, so the grid covers exactly the camera FoV. Row-major, u = col, v = row; `index(row, col)`. - - Tests: len = w·h, all z exact, corners (project with full-res camera lands on (i + 0.5)/s − 0.5), height (64 → 36), left/right symmetry. +- `geometry/scan_grid.rs`: `ScanGrid { width, height: usize, points: Vec, viewport: Viewport }` (private, getters). `new(camera, width, z)` unprojects every pixel center of the `scaled` camera at depth z, so the grid covers exactly the camera FoV. Row-major, u = col, v = row; `index(row, col)`. `with_margin(camera, width, z, margin_rad)`: extra cells on every side continuing the pixel raster (`unproject` with coordinates outside the image), per axis `margin_cells(f, c, n, margin)` = f·(tan(outer) − tan(edge)) with outer = min(edge + margin, `MAX_OFF_AXIS_DEG` 55°); `Viewport { col, row, width, height }` = the visible cells (`full(w, h)`, `contains(col, row)` for sub-cell positions, `crop(full, full_width, out)`). + - Tests: len = w·h, all z exact, corners (project with full-res camera lands on (i + 0.5)/s − 0.5), height (64 → 36), left/right symmetry, margin keeps the visible cells and continues the raster, cap (the 110° test camera gets ≤ 1 column, a huge margin ends at 55°), crop/contains. - `dsp/beamformer.rs`: shared parts of all beamformers. - `Steering { distances: Vec<[f64; M]>, df, grid_size }`, `new(scan_grid, mic_array, f_s, fft_len)`; neither grid nor array is stored. `n_points()`, `grid_size() -> [width, height]` (for CLEAN-SC's clean beam), `vector(point, bin)` = a(r, f_k) via `steering_from_distances`, `gains(point)` = g_m = 1/r_m². Place for steering-vector caching later. - `trait Beamformer { fn n_points(&self); fn power_map(&self, cov, bins, out) }`. `&self` (keeps `rayon` over points possible; scratch allocations per map are negligible). + - `coherent_and_received_power(cov, bins, &steering, point)` and `source_fraction` (their ratio): the power arriving coherently from the point over `bins`, and the mean mic auto-power: DAS with diagonal removal, c = (Σg/M)/((Σg)² − Σg²) so that R = S·a aᴴ gives S·Σg/M (the source's mean mic power). ≈ 1 for a lone source at the point, the sidelobe level for a source elsewhere, ≈ 0 (± fluctuation, also negative) for noise uncorrelated between the mics at any level (≈ −25 dB at the map maximum with memory 40). Beamformer-independent, so it checks the peaks of any map (pipeline) and gates speech (app). Test: R = a aᴴ + (Σg/M)·I gives exactly 1/2 at the source, 0 for the noise alone, less in the corner. - `Normalization` enum lives here (DAS and later functional beamforming use it). - `dsp/beamformer/delay_and_sum.rs`: `Das { normalization: Vec, remove_diag, steering }`, `new(steering, remove_diag, normalization)` precomputes c per point from `steering.gains(i)`. `impl Beamformer`. Siblings: `functional.rs`, `clean_sc.rs`. - `dsp/beamformer/functional.rs`: `Functional { steering, nu, noise_removal, equal_bins, inv_norm2 }` (Dougherty 2014), `new(steering, nu, noise_removal, equal_bins)`. Per bin (bins in parallel) R^{1/ν} = U diag(λ'^{1/ν}) Uᴴ via `symmetric_eigen` + `recompose`, then P_k = (wᴴR^{1/ν}w)^ν / M with w = a/‖a‖. Single source without noise: P = S·‖a_0‖²/M at the source (same as DAS `TrueLocation`); ν = 1 is DAS without diagonal removal. `noise_removal`: λ' = λ − median eigenvalue (noise estimate, assumes < M/2 sources; median, not min, because the noise eigenvalues of an L-frame estimate spread by (1 ± √(M/L))²). λ' below 1e-12·λ_max is set to 0 (`EIGEN_ROUNDING`: the 1/ν power would lift rounding noise to a floor, 1e-16^(1/20) = 0.16). `equal_bins`: each bin map divided by its maximum before summing, bins more than 20 dB (`EQUAL_BINS_GATE_DB`) below the strongest left out; without it low bins (most speech power, no resolution) dominate. Defaults (app): both on, GUI checkboxes. The `Beamformer` impl needs `Const: DimSub` and `DefaultAllocator: Allocator, U1>>` (bounds of `symmetric_eigen`). Tests: hand-built R peak and level at p_0; with 10 dB noise peak at p_0 and fewer points within 3 dB than ν = 1; noise removal gives the noise-free map; equal bins gives a −15 dB source the same peak and drops a −30 dB bin. -- `dsp/beamformer/clean_sc.rs`: `CleanSc { steering, gain, max_iter, stop_ratio, beam_sigma, remove_diag, weight_scale }` (Sijtsma 2007), `new(steering, gain, max_iter, stop_db, beam_sigma, remove_diag)` (asserts ranges; app passes `remove_diag = true`). Weights w = a/(‖a‖√M), √(M − 1) with diagonal removal (levels as DAS `TrueLocation`); G = R, or R with zeroed diagonal; dirty map P_j = wᴴGw. Two passes: all dirty maps first (`DirtyBin`, bins in parallel), threshold = largest dirty value of all bins · 10^(−stop_db/10); then per bin: j_max, source vector h with P_max (h hᴴ − H) w_max = G w_max, H = diag(|h_m|²) (0 without diagonal removal, then h = R w_max / P_max), found by 20 fixed-point steps h = (Gw/P_max + Hw)/√(1 + wᴴHw) (`H_ITER`, as Acoular); clean[j_max] += gain·P_max, G −= gain·P_max (h hᴴ − H), dirty map updated incrementally P_j −= gain·P_max (|w_jᴴh|² − w_jᴴHw_j) (O(points·M)); stops after `max_iter` or below the threshold. The cross-bin threshold keeps bins without a source out (a per-bin threshold added their noise or reflection maximum). Clean maps summed, then separable Gaussian clean beam (`gaussian_blur`, peak 1, σ in grid cells, 0 = single points). Defaults (app): gain 0.9, 10 sources, 20 dB, σ 1.5 cells. Tests: single source with gain 1 is exactly one point (‖a_0‖²/M, with diagonal removal (‖a_0‖⁴ − Σ|a_0m|⁴)/((M − 1)‖a_0‖²)); diagonal removal ignores σ²I 10 dB above the source; a bin 30 dB below the strongest adds nothing at 20 dB but does at 40 dB; two incoherent sources at 1/6 and 5/6 of the width (bin 80, 3840 Hz) give the maximum of each half within one cell; blur keeps the peak and is symmetric. +- `dsp/beamformer/clean_sc.rs`: `CleanSc { steering, gain, max_iter, stop_ratio, beam_sigma, remove_diag, equal_bins, weight_scale }` (Sijtsma 2007), `new(steering, gain, max_iter, stop_db, beam_sigma, remove_diag, equal_bins)` (asserts ranges; app passes `remove_diag = true`). Weights w = a/(‖a‖√M), √(M − 1) with diagonal removal (levels as DAS `TrueLocation`); G = R, or R with zeroed diagonal; dirty map P_j = wᴴGw. Two passes: all dirty maps first (`DirtyBin`, bins in parallel), threshold = largest dirty value of all bins · 10^(−stop_db/10); then per bin: j_max, source vector h with P_max (h hᴴ − H) w_max = G w_max, H = diag(|h_m|²) (0 without diagonal removal, then h = R w_max / P_max), found by 20 fixed-point steps h = (Gw/P_max + Hw)/√(1 + wᴴHw) (`H_ITER`, as Acoular); clean[j_max] += gain·P_max, G −= gain·P_max (h hᴴ − H), dirty map updated incrementally P_j −= gain·P_max (|w_jᴴh|² − w_jᴴHw_j) (O(points·M)); stops after `max_iter` or below the threshold. The cross-bin threshold keeps bins without a source out (a per-bin threshold added their noise or reflection maximum). Clean maps summed (`equal_bins`: each bin's clean map divided by its maximum first, bins that cleaned nothing stay 0; otherwise the loudest bins decide, and those are the low ones, where the array cannot separate a source from its reflection: on hardware CLEAN-SC marked the ceiling reflection of a 3D printer where functional with equal bins marked the printer), then separable Gaussian clean beam (`gaussian_blur`, peak 1, σ in grid cells, 0 = single points). Defaults (app): gain 0.9, 10 sources, 20 dB, σ 1.5 cells, equal bins on (`clean_equal_bins`, checkbox in the settings menu). Tests: single source with gain 1 is exactly one point (‖a_0‖²/M, with diagonal removal (‖a_0‖⁴ − Σ|a_0m|⁴)/((M − 1)‖a_0‖²)); diagonal removal ignores σ²I 10 dB above the source; a bin 30 dB below the strongest adds nothing at 20 dB but does at 40 dB, and with equal bins both sources reach exactly 1; two incoherent sources at 1/6 and 5/6 of the width (bin 80, 3840 Hz) give the maximum of each half within one cell; blur keeps the peak and is symmetric. - `Normalization` (Sarradj 2012, formulations III and IV), P = c·aᴴRa with g_m = 1/r_m², Σg = aᴴa: - `TrueLevel`: c = 1/(Σg)², with diagonal removal 1/((Σg)² − Σg²). P = S exactly at the source, but P(p) = cos²θ·‖a_0‖²/‖a‖²: farther points can exceed S, the peak drifts outward off-axis. - `TrueLocation`: c = 1/(M·Σg), with diagonal removal 1/((M − 1)·Σg) (Acoular's M/(M − 1)). Peak exactly at the source if it lies on the grid plane; symmetric lobes for symmetric sources. Used in `main`. - Diagonal removal subtracts Σ_m |a_m|²·R_mm (= Σ g_m·R_mm, computed from `a` directly), removing σ²I in expectation. P can be negative: clip only in the display (dB floor), never per bin. - `power_map(cov: &[SMatrix] (full spectrum, N/2 + 1), bins: RangeInclusive, out: &mut [f64])`: bins outer, points inner (`enumerate` over normalization and `out`), `steering.vector(i, k)` per (bin, point). Overwrites `out`. - Speed (release, 64×36 grid, ≈ 70 bins, M = 16): ≈ 85 ms per map single-threaded, budget ≈ 0.11 s (`map_every` 10). DAS and functional now run the point loop with `rayon` (`par_iter().zip(out.par_iter_mut()).enumerate()`, bins stay sequential); not measured yet. Linear in points and bins. Further reserve: cache steering vectors per bin (saves the 16 `exp` per (bin, point), ≈ 67 MB). - - Tests (all with `false, TrueLevel`): `hand_built_r_peak_at_source` (R = a_0a_0ᴴ near the corner: P(p_0) = 1, Cauchy-Schwarz P·‖a‖² ≤ ‖a_0‖²), `oracle_r_peak_at_source` (source on the center grid point: argmax at source, P = `sine_bin_power`), `stream_r_peak_at_source` (90 dB mic noise, 0.5 s: argmax at source, P within 2 % of oracle; 110 dB would make the argmax flaky, the map is flat on grid scale). Skeletons with `todo!()` (deferred): `diag_removal_hand_built_r_at_source`, `diag_removal_noise_only_is_zero`, `diag_removal_stream_r_peak_at_source`. + - Tests (all with `false, TrueLevel`): `hand_built_r_peak_at_source` (R = a_0a_0ᴴ near the corner: P(p_0) = 1, Cauchy-Schwarz P·‖a‖² ≤ ‖a_0‖²), `oracle_r_peak_at_source` (source on the center grid point: argmax at source, P = `sine_bin_power`), `stream_r_peak_at_source` (90 dB mic noise, 0.5 s: argmax at source, P within 2 % of oracle; 110 dB would make the argmax flaky, the map is flat on grid scale). Diagonal removal (`true, TrueLevel`): `diag_removal_hand_built_r_at_source` (P(p_0) = 1 with the removal normalization), `diag_removal_noise_only_is_zero` (R = σ²I: 0 everywhere with removal, σ²/Σg without), `diag_removal_stream_r_peak_at_source` (110 dB mic noise, 1 s: oracle map exactly the sine power at the source and its argmax there, stream argmax within one cell (the lobe top is flat on the 192-wide grid), P_est within 5 % and closer to the truth than without removal). - `source/simulator.rs`: `P_REF = 20e-6` (private), `level_to_pressure(level_db) -> p_rms` (dB SPL → Pa). `Simulator { scene, sim_sources: Vec>, mic_noise, n: u64, n_total: u64 }` implementing `FrameSource`. - `SimSource { signal, gain: [f64; M] (1/r_mk), delay: [f64; M] (τ_mk, s) }`, built in `new(scene, mic_array)`; `Scene` stays untouched as ground truth. - `read`: frames = min(buf frames, n_total − n); loop frame → mic → Σ sources gain·eval(t − delay) + one noise draw. `Ok(0)` at end. @@ -183,15 +190,18 @@ Status: simulator and live UMA-16 both run end to end (source → STFT → R̂ Hardware findings (tone test indoors): a pure tone gives several bright spots. Causes: room reflections (coherent with a sine; band noise averages them out), DAS sidelobes of the uniform 4×4 at ≈ −13 dB (visible with a −15 dB display range; −6 dB shows the main lobe only), and a wide band summing noise-only bins for a single tone (narrow the band to the tone). Narrow band + smaller dB range made it much better. Next: +0a. **Mac mini demo fixes: test on the Mac.** (1) Camera: what does the "camera 0: ..." line print (format, fps range), what does the camera tooltip measure ("1920x1080, 30 fps")? If still slow, `camshot` gives the pure capture time. (2) Settings: ⚙ in the top-right corner, faint until hovered; changes land in `sound2view-gui.toml`. (3) Border: no dots on the image edge; tune `center_weight_db`. (4) Noise: hover over dots to see "dB above floor" and "dB coherent above 2 kHz" of noise dots vs. real sources, then set the "Coherence" slider between them (default −8 dB); snaps should still show. +0. **Source and speech gates: test on hardware.** Branch `julian/source-tracks` (merges into `julian/claude`). The noise floor is the quietest map of the last minute (a pause is enough); no dot should appear in silence, a talker or a tone should. The overlay is normalized to the map maximum, so red areas appear in silence too (the maximum is always red); a dot marks a maximum above the floor. Hover over a dot: "+x dB above the noise floor, y dB coherent above 2 kHz". Raise `peaks.min_above_floor_db` (8) if dots appear on nothing, lower it if weak sources are missed. A talker should have y above `speech.min_source_db` (−6), reflections and other tracks below while they talk (leakage sits at the sidelobe level, 3–13 dB down at 2–4 kHz, so −6 dB is a compromise: lower it if the talker stays blue, raise it if other tracks turn green). Also: `hold_ms`/`echo_margin_db` as before, `cargo run --release -- mic 100` writes `track_*.wav` to check beams by ear. 1. Test on hardware: CLEAN-SC with two incoherent sources, `rayon` speedup (map times), mic unplug/replug (status `Lost` → reconnect), camera unplug/replug. 1a. macOS: see "macOS (Mac mini): status, continue here" below. 2. Review the Claude-implemented parts (overlay, sliding covariance, functional, CLEAN-SC, GUI, config, errors). +2a. Scan grid margin on hardware: do the image borders go dark with CLEAN-SC when a source (talker, fan) is just outside the image, especially above/below? Map time (printed) with 104 × 64 cells on the Mac mini; lower `grid_margin_deg` or `grid_width` if it exceeds ≈ 100 ms. `edge_margin_cells` and `center_weight_db` were lowered to 0 and 2 after the first test (margin works on hardware). 3. Camera calibration (principal point, distortion) once the spots are small enough to matter. -4. Small things: rename the scene `config.toml` → `scene.toml`, "Save" button for GUI settings, deferred tests (diagonal removal, `TrueLocation`), unused `Stft::scale`. +4. Small things: rename the scene `config.toml` → `scene.toml`, "Save" button for GUI settings, a `TrueLocation` hand-built test. Error handling (thiserror 2 in core): `CaptureError` (capture), `VideoError` (video), `SceneError` (scene), `PipelineError` (pipeline: `InvalidConfig(String)`, `Source(io::Error)`, `MapHandler(BoxError)`). `pub type BoxError = Box` in `lib.rs` for caller-defined parts (map handlers). All core errors are `Send + Sync` (worker thread → GUI). `ImageSource { type Error; fn next_frame(&mut self) -> Result }`. Config errors are `Result`s: `PipelineConfig::validate(sample_rate)` (called by `Pipeline::new`), app `Config::validate` (called by `Config::load`). Remaining asserts are internal invariants (bin counts, buffer lengths) plus `CleanSc::new`'s parameter ranges (the app validates them first). The probes still return `Box` (CLI only). -Open: diag-removal tests and a `TrueLocation` hand-built test; unused `Stft::scale`; `DB_RANGE` constant, `FilterType::Triangle`; tests for `max_freq`/`validate`, min r_mk check in `Simulator::new`, `Covariance` doc comments, off-center sine reference test; replace plain `16`s in tests; Unicode in doc comments. +Open: a `TrueLocation` hand-built test; `DB_RANGE` constant, `FilterType::Triangle`; tests for `max_freq`/`validate`, min r_mk check in `Simulator::new`, `Covariance` doc comments, off-center sine reference test; replace plain `16`s in tests; Unicode in doc comments. ## macOS (Mac mini): status, continue here diff --git a/Cargo.lock b/Cargo.lock index 003b843..d403f00 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -18,6 +18,15 @@ version = "0.1.10" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "366ffbaa4442f4684d91e2cd7c5ea7c4ed8add41959a31447066e279e432b618" +[[package]] +name = "accesskit" +version = "0.24.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d3b7f7f85a7e5f68090000ed7622545829afd484d210358702ae4cb97dd0c320" +dependencies = [ + "uuid", +] + [[package]] name = "accesskit" version = "0.25.1" @@ -33,7 +42,7 @@ version = "0.21.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "52c182f9c282ac9c5638d876d551d15e5f7d397ec263349a0c6a2b61595dd5e4" dependencies = [ - "accesskit", + "accesskit 0.25.1", "accesskit_consumer", "atspi-common", "phf", @@ -47,7 +56,7 @@ version = "0.39.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8a45e2bb300bea6d5f33a82c3e051c3ddc6f41c209750f68a0acbe991a284344" dependencies = [ - "accesskit", + "accesskit 0.25.1", "hashbrown 0.17.1", ] @@ -57,7 +66,7 @@ version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "28e539d64b27fd4c14de8f78ada8f7c944c092f52f7e49fb63316507a57f6937" dependencies = [ - "accesskit", + "accesskit 0.25.1", "accesskit_consumer", "hashbrown 0.17.1", "objc2 0.5.3", @@ -71,7 +80,7 @@ version = "0.27.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5701624bf6a11efb6d4c8e922bf44126ed96250e5faa9a7a0468c979635839a4" dependencies = [ - "accesskit", + "accesskit 0.25.1", "accesskit_consumer", "hashbrown 0.17.1", "objc2 0.5.3", @@ -85,7 +94,7 @@ version = "0.24.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "202f24df034a7476d07b7f74284de84f6d62aabd858dbe7ee9cad3b7ad6f8f9d" dependencies = [ - "accesskit", + "accesskit 0.25.1", "accesskit_atspi_common", "async-channel", "async-executor", @@ -103,12 +112,12 @@ version = "0.35.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ce63f35d6bdcf59f26b76b3379063f738e6412cef46999cc772d46aa3de35adb" dependencies = [ - "accesskit", + "accesskit 0.25.1", "accesskit_consumer", "hashbrown 0.17.1", "static_assertions", - "windows", - "windows-core", + "windows 0.62.2", + "windows-core 0.62.2", ] [[package]] @@ -117,7 +126,7 @@ version = "0.34.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ee96dd150375c05912d8a473f98c2fb155fb3234fac88292d396548a2460fa77" dependencies = [ - "accesskit", + "accesskit 0.25.1", "accesskit_ios", "accesskit_macos", "accesskit_unix", @@ -196,7 +205,7 @@ dependencies = [ "log", "ndk", "ndk-context", - "ndk-sys", + "ndk-sys 0.6.0+11769913", "num_enum", "thiserror 2.0.21", ] @@ -225,6 +234,15 @@ dependencies = [ "num-traits", ] +[[package]] +name = "arbitrary" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3bc62ac97cc33321f50863d514c3bc38a453947a8f9e781137e47c7401020aed" +dependencies = [ + "derive_arbitrary", +] + [[package]] name = "arboard" version = "3.7.0" @@ -410,6 +428,12 @@ version = "1.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0" +[[package]] +name = "atomic_float" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "628d228f918ac3b82fe590352cc719d30664a0c13ca3a60266fe02c7132d480a" + [[package]] name = "atspi" version = "0.29.0" @@ -453,6 +477,16 @@ version = "1.5.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" +[[package]] +name = "bincode" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "36eaf5d7b090263e8150820482d5d93cd964a81e4019913c972f4edcc6edb740" +dependencies = [ + "serde", + "unty", +] + [[package]] name = "bindgen" version = "0.65.1" @@ -476,15 +510,30 @@ dependencies = [ "which", ] +[[package]] +name = "bit-set" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "08807e080ed7f9d5433fa9b275196cfc35414f66a0c79d864dc51a0d825231a3" +dependencies = [ + "bit-vec 0.8.0", +] + [[package]] name = "bit-set" version = "0.10.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "09ec2f926cc3060f09db9ebc5b52823d85268d24bb917e472c0c4bea35780a7d" dependencies = [ - "bit-vec", + "bit-vec 0.9.1", ] +[[package]] +name = "bit-vec" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7" + [[package]] name = "bit-vec" version = "0.9.1" @@ -502,6 +551,15 @@ name = "bitflags" version = "2.13.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "3ded4057c258ba199e2d26386d3af3780957ecaee6c4ef4041c6b4b8b97c0b06" +dependencies = [ + "serde_core", +] + +[[package]] +name = "block" +version = "0.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0d8c1fef690941d3e7788d328517591fecc684c084084702d6ff1641e993699a" [[package]] name = "block2" @@ -540,6 +598,234 @@ version = "3.20.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649" +[[package]] +name = "burn" +version = "0.17.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "57dc51cb3c3dda2a055a2c41a56b337232c2c85cceef7bf1d0ef0cd9ada30b0b" +dependencies = [ + "burn-autodiff", + "burn-candle", + "burn-core", + "burn-cuda", + "burn-ndarray", + "burn-rocm", + "burn-router", + "burn-wgpu", +] + +[[package]] +name = "burn-autodiff" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a178966322ab7ce71405f1324cdc14f79256d85a47138bbd2c8c4f0056148601" +dependencies = [ + "burn-common", + "burn-tensor", + "derive-new 0.7.0", + "hashbrown 0.15.5", + "log", + "num-traits", + "portable-atomic", + "spin 0.10.1", +] + +[[package]] +name = "burn-candle" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9ed0981b3c1d07e9df0f5bef1042921b6db6e88b5d91916fa5dbdd7f0ca921c3" +dependencies = [ + "burn-tensor", + "candle-core", + "derive-new 0.7.0", + "half", +] + +[[package]] +name = "burn-common" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1c3fae76798ea4dd14e6290b6753eb6235ac28c6ceaf6da35ff8396775d5494d" +dependencies = [ + "cubecl-common", + "rayon", + "serde", +] + +[[package]] +name = "burn-core" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2afa81c868c1a9b3fad25c31176945d0cc5181ba7b77c0456bc05cf57fca975c" +dependencies = [ + "ahash", + "bincode", + "burn-common", + "burn-derive", + "burn-tensor", + "data-encoding", + "derive-new 0.7.0", + "flate2", + "half", + "hashbrown 0.15.5", + "log", + "num-traits", + "portable-atomic-util", + "rand 0.9.5", + "rmp-serde", + "serde", + "serde_json", + "spin 0.10.1", + "uuid", +] + +[[package]] +name = "burn-cubecl" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c547cbe414274ab4022abcc85993e1e41aa7cdccc92395ba5658acfdac285e07" +dependencies = [ + "burn-common", + "burn-ir", + "burn-tensor", + "bytemuck", + "cubecl", + "cubecl-std", + "derive-new 0.7.0", + "futures-lite", + "half", + "hashbrown 0.15.5", + "log", + "num-traits", + "rand 0.9.5", + "serde", + "spin 0.10.1", + "text_placeholder", +] + +[[package]] +name = "burn-cuda" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "995bd0b3f52a4cfe0cfe47c16b40b3fd33285d17a086dd583e5b432074857e02" +dependencies = [ + "burn-cubecl", + "burn-tensor", + "bytemuck", + "cubecl", + "derive-new 0.7.0", + "half", + "log", +] + +[[package]] +name = "burn-derive" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "12e9f07ccc658ef072bce2e996f0c38c80ee4c241598b6557afe1877dd87ae98" +dependencies = [ + "derive-new 0.7.0", + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "burn-ir" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d63629f2c8b82ee52dbb9c18becded5117c2faf57365dc271a55c16d139cd91a" +dependencies = [ + "burn-tensor", + "hashbrown 0.15.5", + "portable-atomic-util", + "serde", +] + +[[package]] +name = "burn-ndarray" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7e883846578e6915e1dbaeeb5bce32cc04cff03e7cb79c5836e1e888bbce974f" +dependencies = [ + "atomic_float", + "burn-autodiff", + "burn-common", + "burn-ir", + "burn-tensor", + "derive-new 0.7.0", + "libm", + "macerator", + "matrixmultiply", + "ndarray", + "num-traits", + "paste", + "portable-atomic-util", + "rand 0.9.5", + "spin 0.10.1", +] + +[[package]] +name = "burn-rocm" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bd39d58202558b65b575921b57bff933845e6171296e2b8faf6a9d3610a344c5" +dependencies = [ + "burn-cubecl", + "burn-tensor", + "bytemuck", + "cubecl", + "derive-new 0.7.0", + "half", + "log", +] + +[[package]] +name = "burn-router" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "22ed8614e180f7a58f77e658bd52e206d2f4a1ee37fcb4665c635ea9da90ea8b" +dependencies = [ + "burn-common", + "burn-ir", + "burn-tensor", + "hashbrown 0.15.5", + "log", + "spin 0.10.1", +] + +[[package]] +name = "burn-tensor" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2a70d1562c0d00083939e34daad61dabebb0f8bc8c250d1ef2f5efc31eb93aaf" +dependencies = [ + "burn-common", + "bytemuck", + "colored", + "cubecl", + "derive-new 0.7.0", + "half", + "hashbrown 0.15.5", + "num-traits", + "rand 0.9.5", + "rand_distr 0.5.1", + "serde", + "serde_bytes", +] + +[[package]] +name = "burn-wgpu" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "215bf0e641a27e17bcd3941a11867dcda411c9cb009488c6b6650c8206437c30" +dependencies = [ + "burn-cubecl", + "burn-tensor", + "cubecl", +] + [[package]] name = "bytemuck" version = "1.25.2" @@ -560,6 +846,12 @@ dependencies = [ "syn 3.0.9", ] +[[package]] +name = "byteorder" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b" + [[package]] name = "byteorder-lite" version = "0.1.0" @@ -622,6 +914,28 @@ dependencies = [ "wayland-client", ] +[[package]] +name = "candle-core" +version = "0.8.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "06ccf5ee3532e66868516d9b315f73aec9f34ea1a37ae98514534d458915dbf1" +dependencies = [ + "byteorder", + "gemm 0.17.1", + "half", + "memmap2", + "num-traits", + "num_cpus", + "rand 0.9.5", + "rand_distr 0.5.1", + "rayon", + "safetensors", + "thiserror 1.0.69", + "ug", + "yoke 0.7.5", + "zip", +] + [[package]] name = "cc" version = "1.10.0" @@ -672,7 +986,7 @@ checksum = "65c35e4b699c7e15ccbe7ee35c005e4fc0a278d22238a2857e6ce2dadeda1b06" dependencies = [ "cfg-if", "cpufeatures", - "rand_core", + "rand_core 0.10.1", ] [[package]] @@ -695,6 +1009,17 @@ dependencies = [ "error-code", ] +[[package]] +name = "codespan-reporting" +version = "0.12.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fe6d2e5af09e8c8ad56c969f2157a3d4238cebc7c55f0a517728c38f7b200f81" +dependencies = [ + "serde", + "termcolor", + "unicode-width", +] + [[package]] name = "codespan-reporting" version = "0.13.1" @@ -715,6 +1040,15 @@ dependencies = [ "bytemuck", ] +[[package]] +name = "colored" +version = "3.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "faf9468729b8cbcea668e36183cb69d317348c2e08e994829fb56ebfdfbaac34" +dependencies = [ + "windows-sys 0.52.0", +] + [[package]] name = "combine" version = "4.6.9" @@ -814,8 +1148,8 @@ dependencies = [ "objc2-core-foundation", "objc2-foundation 0.3.2", "web-sys", - "windows", - "windows-core", + "windows 0.62.2", + "windows-core 0.62.2", ] [[package]] @@ -836,6 +1170,15 @@ dependencies = [ "cfg-if", ] +[[package]] +name = "crossbeam-channel" +version = "0.5.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "98b0cc327b5bc766e7fda9c9260cc0fa81b43a8e240440422dff70788e3f9ef1" +dependencies = [ + "crossbeam-utils", +] + [[package]] name = "crossbeam-deque" version = "0.8.8" @@ -868,44 +1211,427 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "460fbee9c2c2f33933d720630a6a0bac33ba7053db5344fac858d4b8952d77d5" [[package]] -name = "cursor-icon" -version = "1.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f27ae1dd37df86211c42e150270f82743308803d90a6f6e6651cd730d5e1732f" - -[[package]] -name = "dasp_sample" -version = "0.11.0" +name = "cubecl" +version = "0.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0c87e182de0887fd5361989c677c4e8f5000cd9491d6d563161a8f3a5519fc7f" +checksum = "b1e438056cf7c25b3adde38240b89842e1c924b8e914731c82ad81161d23e6ff" +dependencies = [ + "cubecl-core", + "cubecl-cuda", + "cubecl-hip", + "cubecl-linalg", + "cubecl-reduce", + "cubecl-runtime", + "cubecl-std", + "cubecl-wgpu", + "half", +] [[package]] -name = "dispatch" -version = "0.2.0" +name = "cubecl-common" +version = "0.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bd0c93bb4b0c6d9b77f4435b0ae98c24d17f1c45b2ff844c6151a07256ca923b" +checksum = "79251bfc7f067ac9038232fe38a317adc2f31cb2fc3800e69fd409ccac7abc1f" +dependencies = [ + "bytemuck", + "derive-new 0.6.0", + "derive_more", + "dirs", + "embassy-futures", + "futures-lite", + "half", + "hashbrown 0.14.5", + "log", + "num-traits", + "portable-atomic", + "rand 0.9.5", + "sanitize-filename", + "serde", + "serde_json", + "spin 0.9.9", +] [[package]] -name = "dispatch2" -version = "0.3.1" +name = "cubecl-core" +version = "0.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1e0e367e4e7da84520dedcac1901e4da967309406d1e51017ae1abfb97adbd38" +checksum = "b03bf4211cdbd68bb0fb8291e0ed825c13da0d1ac01b7c02dce3cee44a6138be" dependencies = [ "bitflags 2.13.2", - "block2 0.6.2", - "libc", - "objc2 0.6.5", + "bytemuck", + "cubecl-common", + "cubecl-ir", + "cubecl-macros", + "cubecl-runtime", + "derive-new 0.6.0", + "derive_more", + "half", + "hashbrown 0.14.5", + "log", + "num-traits", + "paste", + "serde", + "serde_json", + "variadics_please", ] [[package]] -name = "displaydoc" -version = "0.2.7" +name = "cubecl-cpp" +version = "0.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c6232dd377dcc64799954cbd3a9bb882e9cdc1308ccd87b1c098f1fb2eaf82a8" +checksum = "a5eef85cbcc34be7e25fc9d39edf99ed68559862dbf25c1877ebdf4a9595d31b" dependencies = [ - "proc-macro2", - "quote", - "syn 3.0.9", + "bytemuck", + "cubecl-common", + "cubecl-core", + "cubecl-runtime", + "derive-new 0.6.0", + "half", + "log", +] + +[[package]] +name = "cubecl-cuda" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "71e091e4e3a3900faff440aec4053805ec4456f94f4acc4afe8e6b27519c6d16" +dependencies = [ + "bytemuck", + "cubecl-common", + "cubecl-core", + "cubecl-cpp", + "cubecl-runtime", + "cudarc", + "derive-new 0.6.0", + "half", + "log", + "serde", +] + +[[package]] +name = "cubecl-hip" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c2f8c00207517de61cccdc4ca2724bc1db9dab94840beaf4329e43cead3bc4a" +dependencies = [ + "bytemuck", + "cubecl-common", + "cubecl-core", + "cubecl-cpp", + "cubecl-hip-sys", + "cubecl-runtime", + "derive-new 0.6.0", + "half", + "log", + "paste", +] + +[[package]] +name = "cubecl-hip-sys" +version = "6.4.4348400" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e49f4f0f2ca6942889f2f62780bee6b9629faec3a0e81f5c8dabe36f089c49f9" +dependencies = [ + "libc", + "regex", +] + +[[package]] +name = "cubecl-ir" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e096d77646590f0180ed4ce1aa7df4ecc7219f3c4616e9fe72d93ab63a352855" +dependencies = [ + "cubecl-common", + "cubecl-macros-internal", + "derive_more", + "float-ord", + "fnv", + "half", + "hashbrown 0.14.5", + "num-traits", + "portable-atomic", + "serde", + "variadics_please", +] + +[[package]] +name = "cubecl-linalg" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "75aacf86f6004c274e63589aed55c5edcbcdf1b292eaf4ce2c1688c04c41a194" +dependencies = [ + "bytemuck", + "cubecl-common", + "cubecl-core", + "cubecl-reduce", + "cubecl-runtime", + "cubecl-std", + "half", + "serde", +] + +[[package]] +name = "cubecl-macros" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cd74622b5c8cb161e3f7fa0b2b751784ef89ab45acfa355f511eb2219dde337e" +dependencies = [ + "cubecl-common", + "darling", + "derive-new 0.6.0", + "ident_case", + "prettyplease", + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "cubecl-macros-internal" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6a89898212c1eaba0e2f0dffcadc9790b20b75d2ec8836da084370b043be2623" +dependencies = [ + "darling", + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "cubecl-reduce" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7afbdfe03e7e3ca71f61890ebebc6b4390494204b545e6f6bf51a43755449073" +dependencies = [ + "cubecl-core", + "cubecl-runtime", + "cubecl-std", + "num-traits", + "serde", +] + +[[package]] +name = "cubecl-runtime" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "385234520c9e392382737f32ad372b05f345656eb798ba00b72d2722c68b698c" +dependencies = [ + "async-channel", + "bytemuck", + "cfg-if", + "cfg_aliases", + "cubecl-common", + "cubecl-ir", + "derive-new 0.6.0", + "hashbrown 0.14.5", + "log", + "md5", + "serde", + "serde_json", + "spin 0.9.9", + "variadics_please", + "wasm-bindgen-futures", +] + +[[package]] +name = "cubecl-std" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "38868eea6fdc183feb3c46bcf5e666c78e6cf0ddca2c4f3a877785cc0eabd71e" +dependencies = [ + "cubecl-core", + "cubecl-runtime", + "half", + "serde", +] + +[[package]] +name = "cubecl-wgpu" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "77fa2dcfaa6d75cfbc5ff05cafe99ec4a7fb7c0fa7197917e0fd20f5b90979fe" +dependencies = [ + "async-channel", + "bytemuck", + "cfg-if", + "cfg_aliases", + "cubecl-common", + "cubecl-core", + "cubecl-runtime", + "derive-new 0.6.0", + "derive_more", + "hashbrown 0.14.5", + "log", + "web-time", + "wgpu 25.0.2", +] + +[[package]] +name = "cudarc" +version = "0.13.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "486c221362668c63a1636cfa51463b09574433b39029326cff40864b3ba12b6e" +dependencies = [ + "libloading", +] + +[[package]] +name = "cursor-icon" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f27ae1dd37df86211c42e150270f82743308803d90a6f6e6651cd730d5e1732f" + +[[package]] +name = "darling" +version = "0.20.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc7f46116c46ff9ab3eb1597a45688b6715c6e628b5c133e288e709a29bcb4ee" +dependencies = [ + "darling_core", + "darling_macro", +] + +[[package]] +name = "darling_core" +version = "0.20.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0d00b9596d185e565c2207a0b01f8bd1a135483d02d9b7b0a54b11da8d53412e" +dependencies = [ + "fnv", + "ident_case", + "proc-macro2", + "quote", + "strsim", + "syn 2.0.119", +] + +[[package]] +name = "darling_macro" +version = "0.20.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc34b93ccb385b40dc71c6fceac4b2ad23662c7eeb248cf10d529b7e055b6ead" +dependencies = [ + "darling_core", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "dasp_sample" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c87e182de0887fd5361989c677c4e8f5000cd9491d6d563161a8f3a5519fc7f" + +[[package]] +name = "data-encoding" +version = "2.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4583a4551df46e2792f82ceeac45e850d2e2d5debba0b91f102385cda5b11f06" + +[[package]] +name = "derive-new" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d150dea618e920167e5973d70ae6ece4385b7164e0d799fe7c122dd0a5d912ad" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "derive-new" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2cdc8d50f426189eef89dac62fabfa0abb27d5cc008f25bf4156a0203325becc" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "derive_arbitrary" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1b034bd7d5f032402a2479444dcc6f74e36a03f31854d41680fb240ef682a1ac" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.9", +] + +[[package]] +name = "derive_more" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4a9b99b9cbbe49445b21764dc0625032a89b145a2642e67603e1c936f5458d05" +dependencies = [ + "derive_more-impl", +] + +[[package]] +name = "derive_more-impl" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cb7330aeadfbe296029522e6c40f315320aba36fc43a5b3632f3795348f3bd22" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", + "unicode-xid", +] + +[[package]] +name = "dirs" +version = "5.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "44c45a9d03d6676652bcb5e724c7e988de1acad23a711b5217ab9cbecbec2225" +dependencies = [ + "dirs-sys", +] + +[[package]] +name = "dirs-sys" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "520f05a5cbd335fae5a99ff7a6ab8627577660ee5cfd6a94a6a929b52ff0321c" +dependencies = [ + "libc", + "option-ext", + "redox_users", + "windows-sys 0.48.0", +] + +[[package]] +name = "dispatch" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bd0c93bb4b0c6d9b77f4435b0ae98c24d17f1c45b2ff844c6151a07256ca923b" + +[[package]] +name = "dispatch2" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e0e367e4e7da84520dedcac1901e4da967309406d1e51017ae1abfb97adbd38" +dependencies = [ + "bitflags 2.13.2", + "block2 0.6.2", + "libc", + "objc2 0.6.5", +] + +[[package]] +name = "displaydoc" +version = "0.2.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c6232dd377dcc64799954cbd3a9bb882e9cdc1308ccd87b1c098f1fb2eaf82a8" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.9", ] [[package]] @@ -944,6 +1670,42 @@ version = "1.0.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "92773504d58c093f6de2459af4af33faa518c13451eb8f2b5698ed3d36e7c813" +[[package]] +name = "dyn-stack" +version = "0.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "56e53799688f5632f364f8fb387488dd05db9fe45db7011be066fc20e7027f8b" +dependencies = [ + "bytemuck", + "reborrow", +] + +[[package]] +name = "dyn-stack" +version = "0.13.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1c4713e43e2886ba72b8271aa66c93d722116acf7a75555cce11dcde84388fe8" +dependencies = [ + "bytemuck", + "dyn-stack-macros", +] + +[[package]] +name = "dyn-stack-macros" +version = "0.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e1d926b4d407d372f141f93bb444696142c29d32962ccbd3531117cf3aa0bfa9" + +[[package]] +name = "ecolor" +version = "0.36.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0656a7530b9e2841ee7a5f983965bdd9840c32f4012f759cfd44b912cced6b34" +dependencies = [ + "bytemuck", + "emath 0.36.2", +] + [[package]] name = "ecolor" version = "0.37.0" @@ -951,7 +1713,41 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "31bcbef0d4263a63eb32c4834e5d711613035ce1333468e107d928923e21e6e2" dependencies = [ "bytemuck", - "emath", + "emath 0.37.0", +] + +[[package]] +name = "eframe" +version = "0.36.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "55ad41fac6e3149abb4866e93df37dbd33baa95db525c3368c944584e95a5dc2" +dependencies = [ + "ahash", + "bytemuck", + "document-features", + "egui 0.36.2", + "egui-wgpu 0.36.2", + "egui-winit 0.36.2", + "egui_glow 0.36.2", + "glow 0.17.0", + "glutin", + "glutin-winit", + "image", + "js-sys", + "log", + "objc2 0.6.5", + "objc2-app-kit 0.3.2", + "objc2-foundation 0.3.2", + "parking_lot", + "percent-encoding", + "profiling", + "raw-window-handle", + "static_assertions", + "wasm-bindgen", + "web-sys", + "web-time", + "windows-sys 0.61.2", + "winit", ] [[package]] @@ -963,10 +1759,10 @@ dependencies = [ "ahash", "bytemuck", "document-features", - "egui", - "egui-wgpu", - "egui-winit", - "egui_glow", + "egui 0.37.0", + "egui-wgpu 0.37.0", + "egui-winit 0.37.0", + "egui_glow 0.37.0", "egui_system_font_provider", "glutin", "glutin-winit", @@ -986,22 +1782,42 @@ dependencies = [ "wasm-bindgen", "web-sys", "web-time", - "wgpu", + "wgpu 30.0.1", "windows-sys 0.61.2", "winit", ] +[[package]] +name = "egui" +version = "0.36.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc938cc27cd911415e1e4d151bc56ff9df063d8db081374e87aeb805ea74b2ba" +dependencies = [ + "accesskit 0.24.1", + "ahash", + "bitflags 2.13.2", + "emath 0.36.2", + "epaint 0.36.2", + "itertools", + "log", + "nohash-hasher", + "profiling", + "smallvec", + "unicode-segmentation", + "web-sys", +] + [[package]] name = "egui" version = "0.37.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "bec9144f49b7a28b5f395d3147e325b4fe3e77ba686eac11d1bdf0a7da3f375f" dependencies = [ - "accesskit", + "accesskit 0.25.1", "ahash", "bitflags 2.13.2", - "emath", - "epaint", + "emath 0.37.0", + "epaint 0.37.0", "itertools", "log", "nohash-hasher", @@ -1011,6 +1827,26 @@ dependencies = [ "web-sys", ] +[[package]] +name = "egui-wgpu" +version = "0.36.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5cef1f6d1bba40b09a4d0d62cb822b81030922cfbf65c5373398c3a3caafc5bd" +dependencies = [ + "ahash", + "bytemuck", + "document-features", + "egui 0.36.2", + "epaint 0.36.2", + "log", + "profiling", + "thiserror 2.0.21", + "type-map", + "web-time", + "wgpu 30.0.1", + "winit", +] + [[package]] name = "egui-wgpu" version = "0.37.0" @@ -1020,14 +1856,34 @@ dependencies = [ "ahash", "bytemuck", "document-features", - "egui", - "epaint", + "egui 0.37.0", + "epaint 0.37.0", "log", "profiling", "thiserror 2.0.21", "type-map", "web-time", - "wgpu", + "wgpu 30.0.1", + "winit", +] + +[[package]] +name = "egui-winit" +version = "0.36.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "98466000559d66ba4db786a30448b01483946add4188e7f10f522a27b4fb89cc" +dependencies = [ + "arboard", + "bytemuck", + "egui 0.36.2", + "log", + "objc2 0.6.5", + "objc2-foundation 0.3.2", + "objc2-ui-kit 0.3.2", + "profiling", + "raw-window-handle", + "smithay-clipboard", + "web-time", "winit", ] @@ -1041,7 +1897,7 @@ dependencies = [ "arboard", "block2 0.6.2", "bytemuck", - "egui", + "egui 0.37.0", "log", "objc2 0.6.5", "objc2-app-kit 0.3.2", @@ -1056,6 +1912,20 @@ dependencies = [ "winit", ] +[[package]] +name = "egui_glow" +version = "0.36.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "606ed4167daf341e5753d703285c4c1d562324b2c07aa8f405cd7d066fe58f3b" +dependencies = [ + "bytemuck", + "egui 0.36.2", + "glow 0.17.0", + "log", + "profiling", + "winit", +] + [[package]] name = "egui_glow" version = "0.37.0" @@ -1063,8 +1933,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9275ca5cc35a6b7c94e0e8cedf22542f66cb0cf45ed8ab2657c78ff49fec7268" dependencies = [ "bytemuck", - "egui", - "glow", + "egui 0.37.0", + "glow 0.17.0", "log", "profiling", "winit", @@ -1076,7 +1946,7 @@ version = "0.37.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6f2a2fdfe3c05dd9f691d2b1388b2312d6dce1af502f217c22136d22d9bd76f6" dependencies = [ - "epaint", + "epaint 0.37.0", "fontique", "log", "poll-promise", @@ -1091,6 +1961,15 @@ version = "1.19.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0e9c71c2167ca323c882b99918929403426e2373ea17242ff5653e0d5e1058be" +[[package]] +name = "emath" +version = "0.36.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52c8d4141cf0f60fb2f7aefaf352d3ffa5171ef0ab09731c64391801e95e48f1" +dependencies = [ + "bytemuck", +] + [[package]] name = "emath" version = "0.37.0" @@ -1100,12 +1979,30 @@ dependencies = [ "bytemuck", ] +[[package]] +name = "embassy-futures" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc2d050bdc5c21e0862a89256ed8029ae6c290a93aecefc73084b3002cdebb01" + [[package]] name = "endi" version = "1.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "66b7e2430c6dff6a955451e2cfc438f09cea1965a9d6f87f7e3b90decc014099" +[[package]] +name = "enum-as-inner" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a1e6a265c649f3f5979b601d26f1d05ada116434c87741c9493cb56218f76cbc" +dependencies = [ + "heck", + "proc-macro2", + "quote", + "syn 2.0.119", +] + [[package]] name = "enumflags2" version = "0.7.12" @@ -1127,6 +2024,31 @@ dependencies = [ "syn 2.0.119", ] +[[package]] +name = "epaint" +version = "0.36.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a0ec0308dc23130fc1a7d4623a8534c2ed069c6dc84e258ee4f3de99615aa1a3" +dependencies = [ + "ahash", + "bytemuck", + "ecolor 0.36.2", + "emath 0.36.2", + "epaint_default_fonts 0.36.2", + "font-types", + "harfrust", + "log", + "nohash-hasher", + "parking_lot", + "profiling", + "self_cell", + "skrifa", + "smallvec", + "unicode-general-category", + "unicode-segmentation", + "vello_cpu 0.1.0", +] + [[package]] name = "epaint" version = "0.37.0" @@ -1135,9 +2057,9 @@ checksum = "97bac86ae3dde93fc3951e790951caea739ff39a6c89d909cdaf1d9278dbb1bd" dependencies = [ "ahash", "bytemuck", - "ecolor", - "emath", - "epaint_default_fonts", + "ecolor 0.37.0", + "emath 0.37.0", + "epaint_default_fonts 0.37.0", "font-types", "harfrust", "log", @@ -1150,9 +2072,15 @@ dependencies = [ "unicode-general-category", "unicode-properties", "unicode-segmentation", - "vello_cpu", + "vello_cpu 0.2.0", ] +[[package]] +name = "epaint_default_fonts" +version = "0.36.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "773fa9c96dd0dbef887e39d0ed6177f141cce4d68e6041df77570aa3702dfa13" + [[package]] name = "epaint_default_fonts" version = "0.37.0" @@ -1260,6 +2188,12 @@ dependencies = [ "zlib-rs", ] +[[package]] +name = "float-ord" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8ce81f49ae8a0482e4c55ea62ebbd7e5a686af544c00b9d090bba3ff9be97b3d" + [[package]] name = "flume" version = "0.11.1" @@ -1269,9 +2203,15 @@ dependencies = [ "futures-core", "futures-sink", "nanorand", - "spin", + "spin 0.9.9", ] +[[package]] +name = "fnv" +version = "1.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1" + [[package]] name = "foldhash" version = "0.1.5" @@ -1310,8 +2250,8 @@ dependencies = [ "read-fonts", "roxmltree", "smallvec", - "windows", - "windows-core", + "windows 0.62.2", + "windows-core 0.62.2", "yeslogic-fontconfig-sys", ] @@ -1380,36 +2320,273 @@ dependencies = [ name = "futures-macro" version = "0.3.34" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9fb9654ba8355388abeb8dcb4fc62f511300867002afc858860463bdd9fe0c44" +checksum = "9fb9654ba8355388abeb8dcb4fc62f511300867002afc858860463bdd9fe0c44" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.9", +] + +[[package]] +name = "futures-sink" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1944426bf7d03f1d14f708785e4b33efd750b36d48a157b836b3efc15ede8e1d" + +[[package]] +name = "futures-task" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cd417de3d1d015fc3bfd2b1ea46dfc7bab72ef86f1cc7cc9c78e728b34a6d1fd" + +[[package]] +name = "futures-util" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0d50a92467f8ba5dd6e3ee5d4bd04d73ab2e4e1c44474a0674821dfce14b79bc" +dependencies = [ + "futures-core", + "futures-macro", + "futures-task", + "pin-project-lite", + "slab", +] + +[[package]] +name = "gemm" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ab24cc62135b40090e31a76a9b2766a501979f3070fa27f689c27ec04377d32" +dependencies = [ + "dyn-stack 0.10.0", + "gemm-c32 0.17.1", + "gemm-c64 0.17.1", + "gemm-common 0.17.1", + "gemm-f16 0.17.1", + "gemm-f32 0.17.1", + "gemm-f64 0.17.1", + "num-complex", + "num-traits", + "paste", + "raw-cpuid 10.7.0", + "seq-macro", +] + +[[package]] +name = "gemm" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ab96b703d31950f1aeddded248bc95543c9efc7ac9c4a21fda8703a83ee35451" +dependencies = [ + "dyn-stack 0.13.2", + "gemm-c32 0.18.2", + "gemm-c64 0.18.2", + "gemm-common 0.18.2", + "gemm-f16 0.18.2", + "gemm-f32 0.18.2", + "gemm-f64 0.18.2", + "num-complex", + "num-traits", + "paste", + "raw-cpuid 11.6.0", + "seq-macro", +] + +[[package]] +name = "gemm-c32" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9c030d0b983d1e34a546b86e08f600c11696fde16199f971cd46c12e67512c0" +dependencies = [ + "dyn-stack 0.10.0", + "gemm-common 0.17.1", + "num-complex", + "num-traits", + "paste", + "raw-cpuid 10.7.0", + "seq-macro", +] + +[[package]] +name = "gemm-c32" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f6db9fd9f40421d00eea9dd0770045a5603b8d684654816637732463f4073847" +dependencies = [ + "dyn-stack 0.13.2", + "gemm-common 0.18.2", + "num-complex", + "num-traits", + "paste", + "raw-cpuid 11.6.0", + "seq-macro", +] + +[[package]] +name = "gemm-c64" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fbb5f2e79fefb9693d18e1066a557b4546cd334b226beadc68b11a8f9431852a" +dependencies = [ + "dyn-stack 0.10.0", + "gemm-common 0.17.1", + "num-complex", + "num-traits", + "paste", + "raw-cpuid 10.7.0", + "seq-macro", +] + +[[package]] +name = "gemm-c64" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dfcad8a3d35a43758330b635d02edad980c1e143dc2f21e6fd25f9e4eada8edf" +dependencies = [ + "dyn-stack 0.13.2", + "gemm-common 0.18.2", + "num-complex", + "num-traits", + "paste", + "raw-cpuid 11.6.0", + "seq-macro", +] + +[[package]] +name = "gemm-common" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a2e7ea062c987abcd8db95db917b4ffb4ecdfd0668471d8dc54734fdff2354e8" +dependencies = [ + "bytemuck", + "dyn-stack 0.10.0", + "half", + "num-complex", + "num-traits", + "once_cell", + "paste", + "pulp 0.18.22", + "raw-cpuid 10.7.0", + "rayon", + "seq-macro", + "sysctl 0.5.5", +] + +[[package]] +name = "gemm-common" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a352d4a69cbe938b9e2a9cb7a3a63b7e72f9349174a2752a558a8a563510d0f3" +dependencies = [ + "bytemuck", + "dyn-stack 0.13.2", + "half", + "libm", + "num-complex", + "num-traits", + "once_cell", + "paste", + "pulp 0.21.5", + "raw-cpuid 11.6.0", + "rayon", + "seq-macro", + "sysctl 0.6.0", +] + +[[package]] +name = "gemm-f16" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7ca4c06b9b11952071d317604acb332e924e817bd891bec8dfb494168c7cedd4" +dependencies = [ + "dyn-stack 0.10.0", + "gemm-common 0.17.1", + "gemm-f32 0.17.1", + "half", + "num-complex", + "num-traits", + "paste", + "raw-cpuid 10.7.0", + "rayon", + "seq-macro", +] + +[[package]] +name = "gemm-f16" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cff95ae3259432f3c3410eaa919033cd03791d81cebd18018393dc147952e109" +dependencies = [ + "dyn-stack 0.13.2", + "gemm-common 0.18.2", + "gemm-f32 0.18.2", + "half", + "num-complex", + "num-traits", + "paste", + "raw-cpuid 11.6.0", + "rayon", + "seq-macro", +] + +[[package]] +name = "gemm-f32" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e9a69f51aaefbd9cf12d18faf273d3e982d9d711f60775645ed5c8047b4ae113" dependencies = [ - "proc-macro2", - "quote", - "syn 3.0.9", + "dyn-stack 0.10.0", + "gemm-common 0.17.1", + "num-complex", + "num-traits", + "paste", + "raw-cpuid 10.7.0", + "seq-macro", ] [[package]] -name = "futures-sink" -version = "0.3.34" +name = "gemm-f32" +version = "0.18.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1944426bf7d03f1d14f708785e4b33efd750b36d48a157b836b3efc15ede8e1d" +checksum = "bc8d3d4385393304f407392f754cd2dc4b315d05063f62cf09f47b58de276864" +dependencies = [ + "dyn-stack 0.13.2", + "gemm-common 0.18.2", + "num-complex", + "num-traits", + "paste", + "raw-cpuid 11.6.0", + "seq-macro", +] [[package]] -name = "futures-task" -version = "0.3.34" +name = "gemm-f64" +version = "0.17.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "cd417de3d1d015fc3bfd2b1ea46dfc7bab72ef86f1cc7cc9c78e728b34a6d1fd" +checksum = "aa397a48544fadf0b81ec8741e5c0fba0043008113f71f2034def1935645d2b0" +dependencies = [ + "dyn-stack 0.10.0", + "gemm-common 0.17.1", + "num-complex", + "num-traits", + "paste", + "raw-cpuid 10.7.0", + "seq-macro", +] [[package]] -name = "futures-util" -version = "0.3.34" +name = "gemm-f64" +version = "0.18.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0d50a92467f8ba5dd6e3ee5d4bd04d73ab2e4e1c44474a0674821dfce14b79bc" +checksum = "35b2a4f76ce4b8b16eadc11ccf2e083252d8237c1b589558a49b0183545015bd" dependencies = [ - "futures-core", - "futures-macro", - "futures-task", - "pin-project-lite", - "slab", + "dyn-stack 0.13.2", + "gemm-common 0.18.2", + "num-complex", + "num-traits", + "paste", + "raw-cpuid 11.6.0", + "seq-macro", ] [[package]] @@ -1456,7 +2633,7 @@ dependencies = [ "cfg-if", "libc", "r-efi 6.0.0", - "rand_core", + "rand_core 0.10.1", ] [[package]] @@ -1494,6 +2671,22 @@ version = "0.33.12" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "37bfe73dc8ec21f54d181e5a60554327ec3f1c600c455a75ccbfc65d2106a6c4" +[[package]] +name = "glifo" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed4a1bb24121291d27230c1b1b44e07d6a9b28cefdb32fe1581dfb84e14f940a" +dependencies = [ + "bytemuck", + "foldhash 0.2.0", + "hashbrown 0.17.1", + "log", + "peniko", + "skrifa", + "smallvec", + "vello_common 0.1.0", +] + [[package]] name = "glifo" version = "0.3.0" @@ -1508,7 +2701,7 @@ dependencies = [ "png", "skrifa", "smallvec", - "vello_common", + "vello_common 0.2.0", ] [[package]] @@ -1517,6 +2710,18 @@ version = "0.3.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e4eba85ea1d0a966a983acd07deee566e67395d2d96b6fb39e62b5a833f1eb0b" +[[package]] +name = "glow" +version = "0.16.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c5e5ea60d70410161c8bf5da3fdfeaa1c72ed2c15f8bbb9d19fe3a4fad085f08" +dependencies = [ + "js-sys", + "slotmap", + "wasm-bindgen", + "web-sys", +] + [[package]] name = "glow" version = "0.17.0" @@ -1595,6 +2800,37 @@ dependencies = [ "gl_generator", ] +[[package]] +name = "gpu-alloc" +version = "0.6.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "45cf04b2726f02df5508c6de726acdc90cdf97ac771a9a0ffd8ba10a6e696bf9" +dependencies = [ + "bitflags 2.13.2", + "gpu-alloc-types", +] + +[[package]] +name = "gpu-alloc-types" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b2bbed164dd10ed526c2e4fe3e721ca4a71c61730e5aafac6844b417b3227058" +dependencies = [ + "bitflags 2.13.2", +] + +[[package]] +name = "gpu-allocator" +version = "0.27.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c151a2a5ef800297b4e79efa4f4bec035c5f51d5ae587287c9b952bdf734cacd" +dependencies = [ + "log", + "presser", + "thiserror 1.0.69", + "windows 0.58.0", +] + [[package]] name = "gpu-allocator" version = "0.28.0" @@ -1606,7 +2842,27 @@ dependencies = [ "log", "presser", "thiserror 2.0.21", - "windows", + "windows 0.62.2", +] + +[[package]] +name = "gpu-descriptor" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b89c83349105e3732062a895becfc71a8f921bb71ecbbdd8ff99263e3b53a0ca" +dependencies = [ + "bitflags 2.13.2", + "gpu-descriptor-types", + "hashbrown 0.15.5", +] + +[[package]] +name = "gpu-descriptor-types" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fdf242682df893b86f33a73828fb09ca4b2d3bb6cc95249707fc684d27484b91" +dependencies = [ + "bitflags 2.13.2", ] [[package]] @@ -1624,9 +2880,13 @@ version = "2.7.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6ea2d84b969582b4b1864a92dc5d27cd2b77b622a8d79306834f1be5ba20d84b" dependencies = [ + "bytemuck", "cfg-if", "crunchy", "num-traits", + "rand 0.9.5", + "rand_distr 0.5.1", + "serde", "zerocopy", ] @@ -1642,13 +2902,36 @@ dependencies = [ "smallvec", ] +[[package]] +name = "hashbrown" +version = "0.13.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "43a3c133739dddd0d2990f9a4bdf8eb4b21ef50e4851ca85ab661199821d510e" +dependencies = [ + "ahash", +] + +[[package]] +name = "hashbrown" +version = "0.14.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1" +dependencies = [ + "ahash", + "allocator-api2", + "serde", +] + [[package]] name = "hashbrown" version = "0.15.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1" dependencies = [ + "allocator-api2", + "equivalent", "foldhash 0.1.5", + "serde", ] [[package]] @@ -1671,6 +2954,12 @@ dependencies = [ "foldhash 0.2.0", ] +[[package]] +name = "heck" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" + [[package]] name = "hermit-abi" version = "0.5.3" @@ -1683,6 +2972,12 @@ version = "0.4.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70" +[[package]] +name = "hexf-parse" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dfa686283ad6dd069f105e5ab091b04c62850d3e4cf5d67debad1933f55023df" + [[package]] name = "home" version = "0.5.12" @@ -1692,6 +2987,12 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "hound" +version = "3.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "62adaabb884c94955b19907d60019f4e145d091c75345379e70d1ee696f7854f" + [[package]] name = "icu_collections" version = "2.4.0" @@ -1701,7 +3002,7 @@ dependencies = [ "displaydoc", "potential_utf", "utf8_iter", - "yoke", + "yoke 0.8.4", "zerofrom", "zerovec", ] @@ -1769,12 +3070,18 @@ dependencies = [ "displaydoc", "icu_locale_core", "writeable", - "yoke", + "yoke 0.8.4", "zerofrom", "zerotrie", "zerovec", ] +[[package]] +name = "ident_case" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9e0384b61958566e926dc50660321d12159025e767c18e043daf26b70104c39" + [[package]] name = "idna" version = "1.1.0" @@ -1829,6 +3136,12 @@ dependencies = [ "either", ] +[[package]] +name = "itoa" +version = "1.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" + [[package]] name = "jni" version = "0.22.4" @@ -2028,12 +3341,49 @@ version = "0.4.34" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6" +[[package]] +name = "macerator" +version = "0.2.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "84b0b2dbe8b22f9e96ba12e29964889010117f92e6bd006010887320ae58e2f0" +dependencies = [ + "bytemuck", + "cfg_aliases", + "half", + "macerator-macros", + "moddef", + "num-traits", + "paste", + "rustc_version", +] + +[[package]] +name = "macerator-macros" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed5ce85961d618ce9794bdf822bfe96fe9dd341aa5b033b454f7a8d96e79b9b1" +dependencies = [ + "darling", + "proc-macro2", + "quote", + "syn 2.0.119", +] + [[package]] name = "mach2" version = "0.6.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "dae608c151f68243f2b000364e1f7b186d9c29845f7d2d85bd31b9ad77ad552b" +[[package]] +name = "malloc_buf" +version = "0.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "62bb907fe88d54d8d9ce32a3cceab4218ed2f6b7d35617cafe9adf84e43919cb" +dependencies = [ + "libc", +] + [[package]] name = "matrixmultiply" version = "0.3.11" @@ -2041,9 +3391,18 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "3f607c237553f086e7043417a51df26b2eb899d3caff94e6a67592ff992fedc7" dependencies = [ "autocfg", + "num_cpus", + "once_cell", "rawpointer", + "thread-tree", ] +[[package]] +name = "md5" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "490cc448043f947bae3cbee9c203358d62dbee0db12107a74be5c30ccfd09771" + [[package]] name = "memchr" version = "2.8.3" @@ -2057,6 +3416,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6b0ca1155a9ecc25aa12e8313a119f32edf0255fea2f44839c4ac0a12b2ac157" dependencies = [ "libc", + "stable_deref_trait", ] [[package]] @@ -2068,6 +3428,21 @@ dependencies = [ "autocfg", ] +[[package]] +name = "metal" +version = "0.31.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f569fb946490b5743ad69813cb19629130ce9374034abe31614a36402d18f99e" +dependencies = [ + "bitflags 2.13.2", + "block", + "core-graphics-types", + "foreign-types", + "log", + "objc", + "paste", +] + [[package]] name = "minimal-lexical" version = "0.2.1" @@ -2094,6 +3469,12 @@ dependencies = [ "simd-adler32", ] +[[package]] +name = "moddef" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4a0b3262dc837d2513fe2ef31ff8461352ef932dcca31ba0c0abe33547cf6b9b" + [[package]] name = "moxcms" version = "0.8.1" @@ -2129,6 +3510,31 @@ dependencies = [ "nasm-rs", ] +[[package]] +name = "naga" +version = "25.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2b977c445f26e49757f9aca3631c3b8b836942cb278d69a92e7b80d3b24da632" +dependencies = [ + "arrayvec", + "bit-set 0.8.0", + "bitflags 2.13.2", + "cfg_aliases", + "codespan-reporting 0.12.0", + "half", + "hashbrown 0.15.5", + "hexf-parse", + "indexmap", + "log", + "num-traits", + "once_cell", + "rustc-hash 1.1.0", + "spirv 0.3.0+sdk-1.3.268.0", + "strum", + "thiserror 2.0.21", + "unicode-ident", +] + [[package]] name = "naga" version = "30.0.1" @@ -2136,11 +3542,11 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a616d2fb8c89516ac2723a581f69d6c18576046bed761bd6b305e5618e6ae130" dependencies = [ "arrayvec", - "bit-set", + "bit-set 0.10.0", "bitflags 2.13.2", "cfg-if", "cfg_aliases", - "codespan-reporting", + "codespan-reporting 0.13.1", "half", "hashbrown 0.17.1", "indexmap", @@ -2150,7 +3556,7 @@ dependencies = [ "num-traits", "once_cell", "rustc-hash 1.1.0", - "spirv", + "spirv 0.4.0+sdk-1.4.341.0", "thiserror 2.0.21", "unicode-ident", ] @@ -2206,6 +3612,22 @@ dependencies = [ "log", ] +[[package]] +name = "ndarray" +version = "0.16.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "882ed72dce9365842bf196bdeedf5055305f11fc8c03dee7bb0194a6cad34841" +dependencies = [ + "matrixmultiply", + "num-complex", + "num-integer", + "num-traits", + "portable-atomic", + "portable-atomic-util", + "rawpointer", + "rayon", +] + [[package]] name = "ndk" version = "0.9.0" @@ -2215,7 +3637,7 @@ dependencies = [ "bitflags 2.13.2", "jni-sys 0.3.1", "log", - "ndk-sys", + "ndk-sys 0.6.0+11769913", "num_enum", "raw-window-handle", "thiserror 1.0.69", @@ -2227,6 +3649,15 @@ version = "0.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "27b02d87554356db9e9a873add8782d4ea6e3e58ea071a9adb9a2e8ddb884a8b" +[[package]] +name = "ndk-sys" +version = "0.5.0+25.2.9519653" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8c196769dd60fd4f363e11d948139556a344e79d451aeb2fa2fd040738ef7691" +dependencies = [ + "jni-sys 0.3.1", +] + [[package]] name = "ndk-sys" version = "0.6.0+11769913" @@ -2298,6 +3729,20 @@ dependencies = [ "memchr", ] +[[package]] +name = "num" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "35bd024e8b2ff75562e5f34e7f4905839deb4b22955ef5e73d2fea1b9813cb23" +dependencies = [ + "num-bigint", + "num-complex", + "num-integer", + "num-iter", + "num-rational", + "num-traits", +] + [[package]] name = "num-bigint" version = "0.4.8" @@ -2314,6 +3759,7 @@ version = "0.4.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "73f88a1307638156682bada9d7604135552957b7818057dcef22705b4d509495" dependencies = [ + "bytemuck", "num-traits", "serde", ] @@ -2338,6 +3784,16 @@ dependencies = [ "num-traits", ] +[[package]] +name = "num-iter" +version = "0.1.46" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c92800bd69a1eac91786bcfe9da64a897eb72911b8dc3095decbd07429e8048b" +dependencies = [ + "num-integer", + "num-traits", +] + [[package]] name = "num-rational" version = "0.4.2" @@ -2359,6 +3815,16 @@ dependencies = [ "libm", ] +[[package]] +name = "num_cpus" +version = "1.17.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "91df4bbde75afed763b708b7eee1e8e7651e02d97f6d5dd763e89367e957b23b" +dependencies = [ + "hermit-abi", + "libc", +] + [[package]] name = "num_enum" version = "0.7.6" @@ -2381,6 +3847,15 @@ dependencies = [ "syn 2.0.119", ] +[[package]] +name = "objc" +version = "0.2.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "915b1b472bc21c53464d6c8461c9d3af805ba1ef837e1cac254428f4a77177b1" +dependencies = [ + "malloc_buf", +] + [[package]] name = "objc-sys" version = "0.3.5" @@ -2847,6 +4322,12 @@ version = "1.21.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" +[[package]] +name = "option-ext" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "04744f49eae99ab78e0d5c0b603ab218f515ea8cfe5a456d7629ad883a3b6e7d" + [[package]] name = "orbclient" version = "0.3.55" @@ -2857,6 +4338,15 @@ dependencies = [ "libredox", ] +[[package]] +name = "ordered-float" +version = "4.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7bb71e1b3fa6ca1c61f383464aaf2bb0e2f8e772a1f01d486832464de363b951" +dependencies = [ + "num-traits", +] + [[package]] name = "ordered-float" version = "5.5.0" @@ -3121,6 +4611,9 @@ name = "portable-atomic" version = "1.15.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "05c8b63e8d9609db387f0324918f81d68fe27748f084ef092fb35954d0539a85" +dependencies = [ + "serde", +] [[package]] name = "portable-atomic-util" @@ -3140,6 +4633,15 @@ dependencies = [ "zerovec", ] +[[package]] +name = "ppv-lite86" +version = "0.2.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9" +dependencies = [ + "zerocopy", +] + [[package]] name = "presser" version = "0.3.1" @@ -3189,6 +4691,32 @@ version = "1.0.18" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "3d595e54a326bc53c1c197b32d295e14b169e3cfeaa8dc82b529f947fba6bcf5" +[[package]] +name = "pulp" +version = "0.18.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a0a01a0dc67cf4558d279f0c25b0962bd08fc6dec0137699eae304103e882fe6" +dependencies = [ + "bytemuck", + "libm", + "num-complex", + "reborrow", +] + +[[package]] +name = "pulp" +version = "0.21.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "96b86df24f0a7ddd5e4b95c94fc9ed8a98f1ca94d3b01bdce2824097e7835907" +dependencies = [ + "bytemuck", + "cfg-if", + "libm", + "num-complex", + "reborrow", + "version_check", +] + [[package]] name = "pxfm" version = "0.1.30" @@ -3231,6 +4759,16 @@ version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" +[[package]] +name = "rand" +version = "0.9.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9ef1d0d795eb7d84685bca4f72f3649f064e6641543d3a8c415898726a57b41" +dependencies = [ + "rand_chacha", + "rand_core 0.9.5", +] + [[package]] name = "rand" version = "0.10.3" @@ -3239,7 +4777,26 @@ checksum = "65c9fb96cbc91e3478eaae79a69fcd3f1ae4ad052e471fe6732fff548984b4af" dependencies = [ "chacha20", "getrandom 0.4.3", - "rand_core", + "rand_core 0.10.1", +] + +[[package]] +name = "rand_chacha" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d3022b5f1df60f26e1ffddd6c66e8aa15de382ae63b3a0c1bfc0e4d3e3f325cb" +dependencies = [ + "ppv-lite86", + "rand_core 0.9.5", +] + +[[package]] +name = "rand_core" +version = "0.9.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "76afc826de14238e6e8c374ddcc1fa19e374fd8dd986b0d2af0d02377261d83c" +dependencies = [ + "getrandom 0.3.4", ] [[package]] @@ -3248,6 +4805,16 @@ version = "0.10.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69" +[[package]] +name = "rand_distr" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6a8615d50dcf34fa31f7ab52692afec947c4dd0ab803cc87cb3b0b4570ff7463" +dependencies = [ + "num-traits", + "rand 0.9.5", +] + [[package]] name = "rand_distr" version = "0.6.0" @@ -3255,7 +4822,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4d431c2703ccf129de4d45253c03f49ebb22b97d6ad79ee3ecfc7e3f4862c1d8" dependencies = [ "num-traits", - "rand", + "rand 0.10.3", ] [[package]] @@ -3264,6 +4831,24 @@ version = "0.1.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ca45419789ae5a7899559e9512e58ca889e41f04f1f2445e9f4b290ceccd1d08" +[[package]] +name = "raw-cpuid" +version = "10.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6c297679cb867470fa8c9f67dbba74a78d78e3e98d7cf2b08d6d71540f797332" +dependencies = [ + "bitflags 1.3.2", +] + +[[package]] +name = "raw-cpuid" +version = "11.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "498cd0dc59d73224351ee52a95fee0f1a617a2eae0e7d9d720cc622c73a54186" +dependencies = [ + "bitflags 2.13.2", +] + [[package]] name = "raw-window-handle" version = "0.6.2" @@ -3328,6 +4913,12 @@ dependencies = [ "rustfft", ] +[[package]] +name = "reborrow" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "03251193000f4bd3b042892be858ee50e8b3719f2b08e5833ac4353724632430" + [[package]] name = "redox_syscall" version = "0.4.1" @@ -3355,6 +4946,17 @@ dependencies = [ "bitflags 2.13.2", ] +[[package]] +name = "redox_users" +version = "0.4.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ba009ff324d1fc1b900bd1fdb31564febe58a8ccc8a6fdbb93b543d33b13ca43" +dependencies = [ + "getrandom 0.2.17", + "libredox", + "thiserror 1.0.69", +] + [[package]] name = "regex" version = "1.13.1" @@ -3399,6 +5001,25 @@ dependencies = [ "bytemuck", ] +[[package]] +name = "rmp" +version = "0.8.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4ba8be72d372b2c9b35542551678538b562e7cf86c3315773cae48dfbfe7790c" +dependencies = [ + "num-traits", +] + +[[package]] +name = "rmp-serde" +version = "1.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72f81bee8c8ef9b577d1681a70ebbc962c232461e397b22c208c43c04b67a155" +dependencies = [ + "rmp", + "serde", +] + [[package]] name = "roxmltree" version = "0.21.2" @@ -3484,6 +5105,16 @@ dependencies = [ "bytemuck", ] +[[package]] +name = "safetensors" +version = "0.4.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "44560c11236a6130a46ce36c836a62936dc81ebf8c36a37947423571be0e55b6" +dependencies = [ + "serde", + "serde_json", +] + [[package]] name = "same-file" version = "1.0.6" @@ -3493,6 +5124,16 @@ dependencies = [ "winapi-util", ] +[[package]] +name = "sanitize-filename" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2ed72fbaf78e6f2d41744923916966c4fbe3d7c74e3037a8ee482f1115572603" +dependencies = [ + "lazy_static", + "regex", +] + [[package]] name = "scoped-tls" version = "1.0.1" @@ -3536,6 +5177,12 @@ version = "0.6.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "cd0b0ec5f1c1ca621c432a25813d8d60c88abe6d3e08a3eb9cf37d97a0fe3d73" +[[package]] +name = "seq-macro" +version = "0.3.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1bc711410fbe7399f390ca1c3b60ad0f53f80e95c5eb935e52268a0e2cd49acc" + [[package]] name = "serde" version = "1.0.229" @@ -3546,6 +5193,16 @@ dependencies = [ "serde_derive", ] +[[package]] +name = "serde_bytes" +version = "0.11.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a5d440709e79d88e51ac01c4b72fc6cb7314017bb7da9eeff678aa94c10e3ea8" +dependencies = [ + "serde", + "serde_core", +] + [[package]] name = "serde_core" version = "1.0.229" @@ -3566,6 +5223,19 @@ dependencies = [ "syn 3.0.9", ] +[[package]] +name = "serde_json" +version = "1.0.154" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7e9cc8b1b85264074fbcc02a88680c4096b1e47df8f739dceb03bf482f04bd6" +dependencies = [ + "itoa", + "memchr", + "serde", + "serde_core", + "zmij", +] + [[package]] name = "serde_repr" version = "0.1.21" @@ -3755,11 +5425,14 @@ dependencies = [ name = "sound2view-app" version = "0.1.0" dependencies = [ - "eframe", + "eframe 0.37.0", + "hound", "image", + "rayon", "serde", "sound2view-core", "toml", + "voice-activity-detection", ] [[package]] @@ -3778,8 +5451,8 @@ dependencies = [ "objc2-core-media", "objc2-core-video", "objc2-foundation 0.3.2", - "rand", - "rand_distr", + "rand 0.10.3", + "rand_distr 0.6.0", "rayon", "realfft", "serde", @@ -3794,6 +5467,26 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "3763264f6b73151db08c50ff20d7d8a0b8796e021cdea7ceedad07b80155fa0e" dependencies = [ "lock_api", + "portable-atomic", +] + +[[package]] +name = "spin" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "023a211cb3138dbc438680b32560ad89f699977624c9f8dbb95a47d5b4c07dd3" +dependencies = [ + "lock_api", + "portable-atomic", +] + +[[package]] +name = "spirv" +version = "0.3.0+sdk-1.3.268.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eda41003dc44290527a59b13432d4a0379379fa074b70174882adfbdfd917844" +dependencies = [ + "bitflags 2.13.2", ] [[package]] @@ -3829,6 +5522,34 @@ version = "0.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6637bab7722d379c8b41ba849228d680cc12d0a45ba1fa2b48f2a30577a06731" +[[package]] +name = "strsim" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f" + +[[package]] +name = "strum" +version = "0.26.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8fec0f0aef304996cf250b31b5a10dee7980c85da9d759361292b8bca5a18f06" +dependencies = [ + "strum_macros", +] + +[[package]] +name = "strum_macros" +version = "0.26.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4c6bee85a5a24955dc440386795aa378cd9cf82acd5f764469152d2270e581be" +dependencies = [ + "heck", + "proc-macro2", + "quote", + "rustversion", + "syn 2.0.119", +] + [[package]] name = "syn" version = "2.0.119" @@ -3851,6 +5572,17 @@ dependencies = [ "unicode-ident", ] +[[package]] +name = "synstructure" +version = "0.13.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "728a70f3dbaf5bab7f0c4b1ac8d7ae5ea60a4b5549c8a5914361c99147a709d2" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + [[package]] name = "synstructure" version = "0.14.0" @@ -3871,6 +5603,34 @@ dependencies = [ "libc", ] +[[package]] +name = "sysctl" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec7dddc5f0fee506baf8b9fdb989e242f17e4b11c61dfbb0635b705217199eea" +dependencies = [ + "bitflags 2.13.2", + "byteorder", + "enum-as-inner", + "libc", + "thiserror 1.0.69", + "walkdir", +] + +[[package]] +name = "sysctl" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "01198a2debb237c62b6826ec7081082d951f46dbb64b0e8c7649a452230d1dfc" +dependencies = [ + "bitflags 2.13.2", + "byteorder", + "enum-as-inner", + "libc", + "thiserror 1.0.69", + "walkdir", +] + [[package]] name = "tempfile" version = "3.27.0" @@ -3893,6 +5653,17 @@ dependencies = [ "winapi-util", ] +[[package]] +name = "text_placeholder" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dd5008f74a09742486ef0047596cf35df2b914e2a8dca5727fcb6ba6842a766b" +dependencies = [ + "hashbrown 0.13.2", + "serde", + "serde_json", +] + [[package]] name = "thiserror" version = "1.0.69" @@ -3933,6 +5704,15 @@ dependencies = [ "syn 3.0.9", ] +[[package]] +name = "thread-tree" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ffbd370cb847953a25954d9f63e14824a36113f8c72eecf6eccef5dc4b45d630" +dependencies = [ + "crossbeam-channel", +] + [[package]] name = "tiff" version = "0.11.3" @@ -4127,6 +5907,27 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "ug" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "03719c61a91b51541f076dfdba45caacf750b230cefaa4b32d6f5411c3f7f437" +dependencies = [ + "gemm 0.18.2", + "half", + "libloading", + "memmap2", + "num", + "num-traits", + "num_cpus", + "rayon", + "safetensors", + "serde", + "thiserror 1.0.69", + "tracing", + "yoke 0.7.5", +] + [[package]] name = "unicode-general-category" version = "1.1.0" @@ -4163,6 +5964,18 @@ version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b4ac048d71ede7ee76d585517add45da530660ef4390e49b098733c6e897f254" +[[package]] +name = "unicode-xid" +version = "0.2.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853" + +[[package]] +name = "unty" +version = "0.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6d49784317cd0d1ee7ec5c716dd598ec5b4483ea832a2dced265471cc0f690ae" + [[package]] name = "url" version = "2.5.8" @@ -4196,20 +6009,46 @@ dependencies = [ name = "v4l" version = "0.14.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d8fbfea44a46799d62c55323f3c55d06df722fbe577851d848d328a1041c3403" +checksum = "d8fbfea44a46799d62c55323f3c55d06df722fbe577851d848d328a1041c3403" +dependencies = [ + "bitflags 1.3.2", + "libc", + "v4l2-sys-mit", +] + +[[package]] +name = "v4l2-sys-mit" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6779878362b9bacadc7893eac76abe69612e8837ef746573c4a5239daf11990b" +dependencies = [ + "bindgen", +] + +[[package]] +name = "variadics_please" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41b6d82be61465f97d42bd1d15bf20f3b0a3a0905018f38f9d6f6962055b0b5c" dependencies = [ - "bitflags 1.3.2", - "libc", - "v4l2-sys-mit", + "proc-macro2", + "quote", + "syn 2.0.119", ] [[package]] -name = "v4l2-sys-mit" -version = "0.3.0" +name = "vello_common" +version = "0.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6779878362b9bacadc7893eac76abe69612e8837ef746573c4a5239daf11990b" +checksum = "b2e9aed918117e8152c9eddfd8362d73c465c23f26c44786aa331707b8a64fa2" dependencies = [ - "bindgen", + "bytemuck", + "fearless_simd", + "guillotiere", + "log", + "peniko", + "smallvec", + "thiserror 2.0.21", ] [[package]] @@ -4228,6 +6067,18 @@ dependencies = [ "thiserror 2.0.21", ] +[[package]] +name = "vello_cpu" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ac7349e1f55f6b801c7c277958df4ea53e7f20f21e8014910ad888b2ecda93ea" +dependencies = [ + "bytemuck", + "glifo 0.2.0", + "hashbrown 0.17.1", + "vello_common 0.1.0", +] + [[package]] name = "vello_cpu" version = "0.2.0" @@ -4235,10 +6086,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ace506ca414548966fcf3c283be94eb1cad8254de488791ee698bf438b0890a9" dependencies = [ "bytemuck", - "glifo", + "glifo 0.3.0", "hashbrown 0.17.1", "png", - "vello_common", + "vello_common 0.2.0", ] [[package]] @@ -4247,6 +6098,18 @@ version = "0.9.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" +[[package]] +name = "voice-activity-detection" +version = "0.1.0" +source = "git+https://github.com/Systemscape/voice-activity-detection.git?rev=5f9346de6b1e295613c96559b149bf41acbab9dc#5f9346de6b1e295613c96559b149bf41acbab9dc" +dependencies = [ + "burn", + "eframe 0.36.2", + "nalgebra", + "serde", + "serde_json", +] + [[package]] name = "walkdir" version = "2.5.0" @@ -4505,6 +6368,34 @@ version = "0.1.12" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a28ac98ddc8b9274cb41bb4d9d4d5c425b6020c50c46f25559911905610b4a88" +[[package]] +name = "wgpu" +version = "25.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec8fb398f119472be4d80bc3647339f56eb63b2a331f6a3d16e25d8144197dd9" +dependencies = [ + "arrayvec", + "bitflags 2.13.2", + "cfg_aliases", + "document-features", + "hashbrown 0.15.5", + "js-sys", + "log", + "naga 25.0.1", + "parking_lot", + "portable-atomic", + "profiling", + "raw-window-handle", + "smallvec", + "static_assertions", + "wasm-bindgen", + "wasm-bindgen-futures", + "web-sys", + "wgpu-core 25.0.2", + "wgpu-hal 25.0.2", + "wgpu-types 25.0.0", +] + [[package]] name = "wgpu" version = "30.0.1" @@ -4520,7 +6411,7 @@ dependencies = [ "hashbrown 0.17.1", "js-sys", "log", - "naga", + "naga 30.0.1", "parking_lot", "portable-atomic", "profiling", @@ -4530,9 +6421,40 @@ dependencies = [ "wasm-bindgen", "wasm-bindgen-futures", "web-sys", - "wgpu-core", - "wgpu-hal", - "wgpu-types", + "wgpu-core 30.0.1", + "wgpu-hal 30.0.1", + "wgpu-types 30.0.1", +] + +[[package]] +name = "wgpu-core" +version = "25.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f7b882196f8368511d613c6aeec80655160db6646aebddf8328879a88d54e500" +dependencies = [ + "arrayvec", + "bit-set 0.8.0", + "bit-vec 0.8.0", + "bitflags 2.13.2", + "cfg_aliases", + "document-features", + "hashbrown 0.15.5", + "indexmap", + "log", + "naga 25.0.1", + "once_cell", + "parking_lot", + "portable-atomic", + "profiling", + "raw-window-handle", + "rustc-hash 1.1.0", + "smallvec", + "thiserror 2.0.21", + "wgpu-core-deps-apple 25.0.0", + "wgpu-core-deps-emscripten 25.0.0", + "wgpu-core-deps-windows-linux-android 25.0.0", + "wgpu-hal 25.0.2", + "wgpu-types 25.0.0", ] [[package]] @@ -4542,8 +6464,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "14c018fce9b6270aa203c2fdd56f3cce996713534bd757e4ea58c8560b121f14" dependencies = [ "arrayvec", - "bit-set", - "bit-vec", + "bit-set 0.10.0", + "bit-vec 0.9.1", "bitflags 2.13.2", "bytemuck", "cfg_aliases", @@ -4551,7 +6473,7 @@ dependencies = [ "hashbrown 0.17.1", "indexmap", "log", - "naga", + "naga 30.0.1", "naga-types", "once_cell", "parking_lot", @@ -4561,13 +6483,22 @@ dependencies = [ "rustc-hash 1.1.0", "smallvec", "thiserror 2.0.21", - "wgpu-core-deps-apple", - "wgpu-core-deps-emscripten", + "wgpu-core-deps-apple 30.0.1", + "wgpu-core-deps-emscripten 30.0.1", "wgpu-core-deps-wasm", - "wgpu-core-deps-windows-linux-android", - "wgpu-hal", + "wgpu-core-deps-windows-linux-android 30.0.1", + "wgpu-hal 30.0.1", "wgpu-naga-bridge", - "wgpu-types", + "wgpu-types 30.0.1", +] + +[[package]] +name = "wgpu-core-deps-apple" +version = "25.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cfd488b3239b6b7b185c3b045c39ca6bf8af34467a4c5de4e0b1a564135d093d" +dependencies = [ + "wgpu-hal 25.0.2", ] [[package]] @@ -4576,7 +6507,16 @@ version = "30.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "061f3d319a40d39d00b1ecc2c33b89fe21d4e6fe01859df3500a3a8ecccd6b68" dependencies = [ - "wgpu-hal", + "wgpu-hal 30.0.1", +] + +[[package]] +name = "wgpu-core-deps-emscripten" +version = "25.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f09ad7aceb3818e52539acc679f049d3475775586f3f4e311c30165cf2c00445" +dependencies = [ + "wgpu-hal 25.0.2", ] [[package]] @@ -4585,7 +6525,7 @@ version = "30.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d98b86cf4abf524a902dd35f18ca6a3f08fc2ae9847c8f10b48e30491b1f0b86" dependencies = [ - "wgpu-hal", + "wgpu-hal 30.0.1", ] [[package]] @@ -4594,7 +6534,16 @@ version = "30.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d260ca2ca8adc654f513a869bf3cac0b47f5b8d3f87803d6942015f8de597aae" dependencies = [ - "wgpu-hal", + "wgpu-hal 30.0.1", +] + +[[package]] +name = "wgpu-core-deps-windows-linux-android" +version = "25.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cba5fb5f7f9c98baa7c889d444f63ace25574833df56f5b817985f641af58e46" +dependencies = [ + "wgpu-hal 25.0.2", ] [[package]] @@ -4603,7 +6552,54 @@ version = "30.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7586165fd5f6d881cb9ce4bb71f40d6caab2c0f1837e3fc1d9788a197fb6004f" dependencies = [ - "wgpu-hal", + "wgpu-hal 30.0.1", +] + +[[package]] +name = "wgpu-hal" +version = "25.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f968767fe4d3d33747bbd1473ccd55bf0f6451f55d733b5597e67b5deab4ad17" +dependencies = [ + "android_system_properties", + "arrayvec", + "ash", + "bit-set 0.8.0", + "bitflags 2.13.2", + "block", + "bytemuck", + "cfg-if", + "cfg_aliases", + "core-graphics-types", + "glow 0.16.0", + "glutin_wgl_sys", + "gpu-alloc", + "gpu-allocator 0.27.0", + "gpu-descriptor", + "hashbrown 0.15.5", + "js-sys", + "khronos-egl", + "libc", + "libloading", + "log", + "metal", + "naga 25.0.1", + "ndk-sys 0.5.0+25.2.9519653", + "objc", + "ordered-float 4.6.0", + "parking_lot", + "portable-atomic", + "profiling", + "range-alloc", + "raw-window-handle", + "renderdoc-sys", + "smallvec", + "thiserror 2.0.21", + "wasm-bindgen", + "web-sys", + "wgpu-types 25.0.0", + "windows 0.58.0", + "windows-core 0.58.0", ] [[package]] @@ -4615,24 +6611,24 @@ dependencies = [ "android_system_properties", "arrayvec", "ash", - "bit-set", + "bit-set 0.10.0", "bitflags 2.13.2", "block2 0.6.2", "bytemuck", "cfg-if", "cfg_aliases", - "glow", + "glow 0.17.0", "glutin_wgl_sys", - "gpu-allocator", + "gpu-allocator 0.28.0", "hashbrown 0.17.1", "js-sys", "khronos-egl", "libc", "libloading", "log", - "naga", + "naga 30.0.1", "naga-types", - "ndk-sys", + "ndk-sys 0.6.0+11769913", "objc2 0.6.5", "objc2-core-foundation", "objc2-core-graphics", @@ -4640,7 +6636,7 @@ dependencies = [ "objc2-metal 0.3.2", "objc2-quartz-core 0.3.2", "once_cell", - "ordered-float", + "ordered-float 5.5.0", "parking_lot", "portable-atomic", "portable-atomic-util", @@ -4656,10 +6652,10 @@ dependencies = [ "wayland-sys", "web-sys", "wgpu-naga-bridge", - "wgpu-types", - "windows", - "windows-core", - "windows-result", + "wgpu-types 30.0.1", + "windows 0.62.2", + "windows-core 0.62.2", + "windows-result 0.4.1", ] [[package]] @@ -4668,8 +6664,22 @@ version = "30.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d2f62e73117bb7a62bfd9c5a5841438a823f6566c6442a808ee269d2d055c081" dependencies = [ - "naga", - "wgpu-types", + "naga 30.0.1", + "wgpu-types 30.0.1", +] + +[[package]] +name = "wgpu-types" +version = "25.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2aa49460c2a8ee8edba3fca54325540d904dd85b2e086ada762767e17d06e8bc" +dependencies = [ + "bitflags 2.13.2", + "bytemuck", + "js-sys", + "log", + "thiserror 2.0.21", + "web-sys", ] [[package]] @@ -4719,6 +6729,16 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "windows" +version = "0.58.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dd04d41d93c4992d421894c18c8b43496aa748dd4c081bac0dc93eb0489272b6" +dependencies = [ + "windows-core 0.58.0", + "windows-targets 0.52.6", +] + [[package]] name = "windows" version = "0.62.2" @@ -4726,7 +6746,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "527fadee13e0c05939a6a05d5bd6eec6cd2e3dbd648b9f8e447c6518133d8580" dependencies = [ "windows-collections", - "windows-core", + "windows-core 0.62.2", "windows-future", "windows-numerics", ] @@ -4737,7 +6757,20 @@ version = "0.3.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "23b2d95af1a8a14a3c7367e1ed4fc9c20e0a26e79551b1454d72583c97cc6610" dependencies = [ - "windows-core", + "windows-core 0.62.2", +] + +[[package]] +name = "windows-core" +version = "0.58.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ba6d44ec8c2591c134257ce647b7ea6b20335bf6379a27dac5f1641fcf59f99" +dependencies = [ + "windows-implement 0.58.0", + "windows-interface 0.58.0", + "windows-result 0.2.0", + "windows-strings 0.1.0", + "windows-targets 0.52.6", ] [[package]] @@ -4746,11 +6779,11 @@ version = "0.62.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb" dependencies = [ - "windows-implement", - "windows-interface", + "windows-implement 0.60.2", + "windows-interface 0.59.3", "windows-link", - "windows-result", - "windows-strings", + "windows-result 0.4.1", + "windows-strings 0.5.1", ] [[package]] @@ -4759,11 +6792,22 @@ version = "0.3.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e1d6f90251fe18a279739e78025bd6ddc52a7e22f921070ccdc67dde84c605cb" dependencies = [ - "windows-core", + "windows-core 0.62.2", "windows-link", "windows-threading", ] +[[package]] +name = "windows-implement" +version = "0.58.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2bbd5b46c938e506ecbce286b6628a02171d56153ba733b6c741fc627ec9579b" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + [[package]] name = "windows-implement" version = "0.60.2" @@ -4775,6 +6819,17 @@ dependencies = [ "syn 2.0.119", ] +[[package]] +name = "windows-interface" +version = "0.58.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "053c4c462dc91d3b1504c6fe5a726dd15e216ba718e84a0e46a88fbe5ded3515" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + [[package]] name = "windows-interface" version = "0.59.3" @@ -4798,10 +6853,19 @@ version = "0.3.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6e2e40844ac143cdb44aead537bbf727de9b044e107a0f1220392177d15b0f26" dependencies = [ - "windows-core", + "windows-core 0.62.2", "windows-link", ] +[[package]] +name = "windows-result" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d1043d8214f791817bab27572aaa8af63732e11bf84aa21a45a78d6c317ae0e" +dependencies = [ + "windows-targets 0.52.6", +] + [[package]] name = "windows-result" version = "0.4.1" @@ -4811,6 +6875,16 @@ dependencies = [ "windows-link", ] +[[package]] +name = "windows-strings" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4cd9b125c486025df0eabcb585e62173c6c9eddcec5d117d3b6e8c30e2ee4d10" +dependencies = [ + "windows-result 0.2.0", + "windows-targets 0.52.6", +] + [[package]] name = "windows-strings" version = "0.5.1" @@ -4820,13 +6894,22 @@ dependencies = [ "windows-link", ] +[[package]] +name = "windows-sys" +version = "0.48.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "677d2418bec65e3338edb076e806bc1ec15693c5d0104683f2efe857f61056a9" +dependencies = [ + "windows-targets 0.48.5", +] + [[package]] name = "windows-sys" version = "0.52.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "282be5f36a8ce781fad8c8ae18fa3f9beff57ec1b52cb3de0789201425d9a33d" dependencies = [ - "windows-targets", + "windows-targets 0.52.6", ] [[package]] @@ -4835,7 +6918,7 @@ version = "0.59.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1e38bc4d79ed67fd075bcc251a1c39b32a1776bbe92e5bef1f0bf1f8c531853b" dependencies = [ - "windows-targets", + "windows-targets 0.52.6", ] [[package]] @@ -4847,20 +6930,35 @@ dependencies = [ "windows-link", ] +[[package]] +name = "windows-targets" +version = "0.48.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9a2fa6e2155d7247be68c096456083145c183cbbbc2764150dda45a87197940c" +dependencies = [ + "windows_aarch64_gnullvm 0.48.5", + "windows_aarch64_msvc 0.48.5", + "windows_i686_gnu 0.48.5", + "windows_i686_msvc 0.48.5", + "windows_x86_64_gnu 0.48.5", + "windows_x86_64_gnullvm 0.48.5", + "windows_x86_64_msvc 0.48.5", +] + [[package]] name = "windows-targets" version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973" dependencies = [ - "windows_aarch64_gnullvm", - "windows_aarch64_msvc", - "windows_i686_gnu", + "windows_aarch64_gnullvm 0.52.6", + "windows_aarch64_msvc 0.52.6", + "windows_i686_gnu 0.52.6", "windows_i686_gnullvm", - "windows_i686_msvc", - "windows_x86_64_gnu", - "windows_x86_64_gnullvm", - "windows_x86_64_msvc", + "windows_i686_msvc 0.52.6", + "windows_x86_64_gnu 0.52.6", + "windows_x86_64_gnullvm 0.52.6", + "windows_x86_64_msvc 0.52.6", ] [[package]] @@ -4872,18 +6970,36 @@ dependencies = [ "windows-link", ] +[[package]] +name = "windows_aarch64_gnullvm" +version = "0.48.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2b38e32f0abccf9987a4e3079dfb67dcd799fb61361e53e2882c3cbaf0d905d8" + [[package]] name = "windows_aarch64_gnullvm" version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3" +[[package]] +name = "windows_aarch64_msvc" +version = "0.48.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc35310971f3b2dbbf3f0690a219f40e2d9afcf64f9ab7cc1be722937c26b4bc" + [[package]] name = "windows_aarch64_msvc" version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469" +[[package]] +name = "windows_i686_gnu" +version = "0.48.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a75915e7def60c94dcef72200b9a8e58e5091744960da64ec734a6c6e9b3743e" + [[package]] name = "windows_i686_gnu" version = "0.52.6" @@ -4896,24 +7012,48 @@ version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66" +[[package]] +name = "windows_i686_msvc" +version = "0.48.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f55c233f70c4b27f66c523580f78f1004e8b5a8b659e05a4eb49d4166cca406" + [[package]] name = "windows_i686_msvc" version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66" +[[package]] +name = "windows_x86_64_gnu" +version = "0.48.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "53d40abd2583d23e4718fddf1ebec84dbff8381c07cae67ff7768bbf19c6718e" + [[package]] name = "windows_x86_64_gnu" version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78" +[[package]] +name = "windows_x86_64_gnullvm" +version = "0.48.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b7b52767868a23d5bab768e390dc5f5c55825b6d30b86c844ff2dc7414044cc" + [[package]] name = "windows_x86_64_gnullvm" version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d" +[[package]] +name = "windows_x86_64_msvc" +version = "0.48.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed94fce61571a4006852b7389a063ab983c02eb1bb37b47f8272ce92d06d9538" + [[package]] name = "windows_x86_64_msvc" version = "0.52.6" @@ -5085,6 +7225,18 @@ dependencies = [ "pkg-config", ] +[[package]] +name = "yoke" +version = "0.7.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "120e6aef9aa629e3d4f52dc8cc43a015c7724194c97dfaf45180d2daf2b77f40" +dependencies = [ + "serde", + "stable_deref_trait", + "yoke-derive 0.7.5", + "zerofrom", +] + [[package]] name = "yoke" version = "0.8.4" @@ -5092,10 +7244,22 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "417ef3fcf6c300b7dbd25c85531a746931c02ad536f3cf4786ed1e88223340b3" dependencies = [ "stable_deref_trait", - "yoke-derive", + "yoke-derive 0.8.4", "zerofrom", ] +[[package]] +name = "yoke-derive" +version = "0.7.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2380878cad4ac9aac1e2435f3eb4020e8374b5f13c296cb75b4620ff8e229154" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", + "synstructure 0.13.2", +] + [[package]] name = "yoke-derive" version = "0.8.4" @@ -5105,7 +7269,7 @@ dependencies = [ "proc-macro2", "quote", "syn 3.0.9", - "synstructure", + "synstructure 0.14.0", ] [[package]] @@ -5252,7 +7416,7 @@ dependencies = [ "proc-macro2", "quote", "syn 3.0.9", - "synstructure", + "synstructure 0.14.0", ] [[package]] @@ -5262,7 +7426,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4ea269c3bd32f0a32c321907a2ae912ba6f4649bb0fc764a15627e99a7095a3f" dependencies = [ "displaydoc", - "yoke", + "yoke 0.8.4", "zerofrom", ] @@ -5272,7 +7436,7 @@ version = "0.11.9" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f58a2799cfe9a1670c7ea8edbd52ef8edb93cb37a7f83bf31ba9e35d9dae6a40" dependencies = [ - "yoke", + "yoke 0.8.4", "zerofrom", "zerovec-derive", ] @@ -5288,12 +7452,32 @@ dependencies = [ "syn 3.0.9", ] +[[package]] +name = "zip" +version = "1.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1b7a5a9285bd4ee13bdeb3f8a4917eb46557e53f270c783849db8bef37b0ad00" +dependencies = [ + "arbitrary", + "crc32fast", + "crossbeam-utils", + "displaydoc", + "indexmap", + "thiserror 1.0.69", +] + [[package]] name = "zlib-rs" version = "0.6.8" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b268e58e7c693d7c271f93ffc4ba3b380412554231c85bf61ca7af91042a4112" +[[package]] +name = "zmij" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b" + [[package]] name = "zune-core" version = "0.5.3" diff --git a/README.md b/README.md index 4e26b23..01e2b9d 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,7 @@ # sound2view +sound2view logo + Acoustic camera with the miniDSP UMA-16 v2. ## Linux setup @@ -44,7 +46,12 @@ cargo run --release -- mic # live maps from the UMA-16 cargo run --release -- scene.toml # simulated scene (default: config.toml) ``` -Settings (audio device, camera name and format, field of view, band, beamformer, overlay range) -are read from `sound2view.toml` in the working directory; every entry is optional and defaults -to the values in the shipped file. Beamformer, nu, band and overlay range can also be changed -live in the GUI (not written back to the file). +Settings (audio device, camera name and format, field of view, band, beamformer, overlay range, +scan grid) are read from `sound2view.toml` in the working directory; every entry is optional and +defaults to the values in the shipped file, which documents each one. Beamformer, band, overlay +range and the source thresholds can also be changed live in the settings menu of the GUI; those +changes are saved to `sound2view-gui.toml` next to it. + +The scan grid extends `pipeline.grid_margin_deg` (15) degrees beyond the camera image on every +side, so a sound source just outside the image gets its own spot there instead of lighting up the +image border; with CLEAN-SC the visible part stays dark. Set it to 0 to save map time. diff --git a/assets/README.md b/assets/README.md new file mode 100644 index 0000000..56f3f75 --- /dev/null +++ b/assets/README.md @@ -0,0 +1,10 @@ +# Logo + +- `logo.svg`: the logo (4×4 microphone array as a heat map in a camera viewfinder). +- `logo-small.svg`: simplified version for 32 px and below (favicon, small icons). +- `logo-wordmark.svg`: logo with the name, e.g. for a README banner or slides. +- `logo-{512,256,128,64}.png` from `logo.svg`, `logo-{32,16}.png` from `logo-small.svg`, rendered + with resvg. `logo-256.png` is the GUI window icon (`include_bytes!` in `crates/app/src/main.rs`). + +Colors match the GUI theme (`crates/app/src/theme.rs`): background #181b1f, amber #f5a524, +frame #e6e8eb, inactive mics #4a5058. diff --git a/assets/logo-128.png b/assets/logo-128.png new file mode 100644 index 0000000..13f9436 Binary files /dev/null and b/assets/logo-128.png differ diff --git a/assets/logo-16.png b/assets/logo-16.png new file mode 100644 index 0000000..15d9fdc Binary files /dev/null and b/assets/logo-16.png differ diff --git a/assets/logo-256.png b/assets/logo-256.png new file mode 100644 index 0000000..153e04f Binary files /dev/null and b/assets/logo-256.png differ diff --git a/assets/logo-32.png b/assets/logo-32.png new file mode 100644 index 0000000..4349d9a Binary files /dev/null and b/assets/logo-32.png differ diff --git a/assets/logo-512.png b/assets/logo-512.png new file mode 100644 index 0000000..097efe8 Binary files /dev/null and b/assets/logo-512.png differ diff --git a/assets/logo-64.png b/assets/logo-64.png new file mode 100644 index 0000000..86d3ec9 Binary files /dev/null and b/assets/logo-64.png differ diff --git a/assets/logo-small.svg b/assets/logo-small.svg new file mode 100644 index 0000000..0236ce5 --- /dev/null +++ b/assets/logo-small.svg @@ -0,0 +1,11 @@ + + sound2view + + + + + + + + + diff --git a/assets/logo-wordmark.svg b/assets/logo-wordmark.svg new file mode 100644 index 0000000..a0bdf8f --- /dev/null +++ b/assets/logo-wordmark.svg @@ -0,0 +1,31 @@ + + sound2view + + + + + + + + + + + + + + + + + + + + + + + + + + + sound2view + diff --git a/assets/logo.svg b/assets/logo.svg new file mode 100644 index 0000000..86c43f6 --- /dev/null +++ b/assets/logo.svg @@ -0,0 +1,28 @@ + + sound2view + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/crates/app/Cargo.toml b/crates/app/Cargo.toml index bd134f2..1ca9f1a 100644 --- a/crates/app/Cargo.toml +++ b/crates/app/Cargo.toml @@ -11,5 +11,9 @@ path = "src/main.rs" sound2view-core = { path = "../core" } image = { version = "0.25.10", default-features = false, features = ["png"] } eframe = "0.37.0" +rayon = "1.12.0" +hound = "3.5.1" +# Speech classifier (lib: Preprocessor + Model); its package also builds burn and eframe 0.36 +voice-activity-detection = { git = "https://github.com/Systemscape/voice-activity-detection.git", rev = "5f9346de6b1e295613c96559b149bf41acbab9dc" } serde = { version = "1.0.229", default-features = false, features = ["derive", "std"] } -toml = { version = "1.1.8", default-features = false, features = ["std", "serde", "parse"] } +toml = { version = "1.1.8", default-features = false, features = ["std", "serde", "parse", "display"] } diff --git a/crates/app/src/config.rs b/crates/app/src/config.rs index 3d41f38..93d053c 100644 --- a/crates/app/src/config.rs +++ b/crates/app/src/config.rs @@ -3,13 +3,16 @@ use std::{error::Error, fs, io}; -use serde::Deserialize; +use serde::{Deserialize, Serialize}; use sound2view_core::{ geometry::camera::{Camera, Intrinsics, Mount}, peaks::PeakConfig, pipeline::PipelineConfig, + tracking::TrackerConfig, }; +use crate::speech::SpeechConfig; + /// Config file, looked up in the working directory pub const CONFIG_PATH: &str = "sound2view.toml"; @@ -22,6 +25,8 @@ pub struct Config { pub beamformer: BeamformerConfig, pub display: DisplayConfig, pub peaks: PeakConfig, + pub tracking: TrackerConfig, + pub speech: SpeechConfig, } impl Config { @@ -137,7 +142,7 @@ impl CameraConfig { } /// Beamforming method -#[derive(Clone, Copy, Debug, PartialEq, Eq, Deserialize)] +#[derive(Clone, Copy, Debug, PartialEq, Eq, Deserialize, Serialize)] #[serde(rename_all = "snake_case")] pub enum Method { /// Delay and sum with diagonal removal and true-location normalization @@ -178,7 +183,7 @@ impl Method { } } -#[derive(Clone, Debug, PartialEq, Deserialize)] +#[derive(Clone, Debug, PartialEq, Deserialize, Serialize)] #[serde(default, deny_unknown_fields)] pub struct BeamformerConfig { pub method: Method, @@ -197,12 +202,15 @@ pub struct BeamformerConfig { pub clean_stop_db: f64, /// CLEAN-SC: standard deviation of the Gaussian clean beam in grid cells (0: single points) pub clean_beam_cells: f64, + /// CLEAN-SC: divide each bin's clean map by its maximum before summing (every bin with a + /// source has the same say; otherwise the loudest, lowest bins decide) + pub clean_equal_bins: bool, } impl Default for BeamformerConfig { fn default() -> Self { Self { - method: Method::Functional, + method: Method::CleanSc, nu: 20.0, functional_noise_removal: true, functional_equal_bins: true, @@ -210,6 +218,7 @@ impl Default for BeamformerConfig { clean_max_iter: 10, clean_stop_db: 20.0, clean_beam_cells: 1.5, + clean_equal_bins: true, } } } @@ -265,6 +274,8 @@ impl Config { "peaks: range_db must be > 0, min_dist_cells >= 0, max_count >= 1".to_string(), ); } + self.tracking.validate()?; + self.speech.validate()?; if self.display.db_range <= 0.0 { return Err("display.db_range must be > 0".to_string()); } @@ -273,3 +284,15 @@ impl Config { .map_err(|e| e.to_string()) } } + +#[cfg(test)] +mod shipped_config_test { + use super::Config; + + /// The example `sound2view.toml` in the repository root parses and is valid. + #[test] + fn shipped_config_is_valid() { + let config: Config = toml::from_str(include_str!("../../../sound2view.toml")).unwrap(); + config.validate().unwrap(); + } +} diff --git a/crates/app/src/gui.rs b/crates/app/src/gui.rs index d702b75..413cda3 100644 --- a/crates/app/src/gui.rs +++ b/crates/app/src/gui.rs @@ -1,31 +1,39 @@ use std::{ + collections::HashMap, ops::{ControlFlow, RangeInclusive}, sync::{ Arc, Mutex, + atomic::{AtomicBool, AtomicU32, Ordering}, mpsc::{Receiver, TrySendError, sync_channel}, }, thread, - time::Duration, + time::{Duration, Instant}, }; use eframe::egui::{ - self, Align, Align2, Color32, ColorImage, FontId, Layout, Margin, Rect, RichText, Slider, - Stroke, TextureHandle, TextureOptions, pos2, + self, Align2, Color32, ColorImage, FontId, Margin, PopupCloseBehavior, Pos2, Rect, RichText, + Sense, Slider, Stroke, TextureHandle, TextureOptions, + containers::menu::{MenuButton, MenuConfig}, + pos2, vec2, }; use sound2view_core::{ BoxError, ImageSource, VideoFrame, - peaks::{Peak, PeakConfig, find_peaks}, + peaks::PeakConfig, pipeline::{MapLayout, PipelineConfig}, render, source::{ capture::CpalSource, video::{CameraEntry, CameraSource, list_cameras}, }, + tracking::{Track, TrackerConfig}, }; use crate::{ config::{BeamformerConfig, CameraConfig, Config, Method}, - run_configured, theme, + gui_state::{GUI_STATE_PATH, GuiState}, + run_configured, + speech::{SpeechDetector, SpeechStatus}, + theme, }; /// State of the camera or the mic, set by its thread and shown in the GUI @@ -72,13 +80,17 @@ impl DeviceStatus { } } -/// Status dot with short text; the full message as tooltip -fn status_indicator(ui: &mut egui::Ui, status: &DeviceStatus, device: &str) { - ui.label(RichText::new(status.short(device)).color(theme::TEXT_WEAK)) - .on_hover_text(status.message(device)); - let (rect, _) = ui.allocate_exact_size(egui::vec2(10.0, 10.0), egui::Sense::hover()); +/// Status dot with short text; the full message and `detail` (if not empty) as tooltip +fn status_indicator(ui: &mut egui::Ui, status: &DeviceStatus, device: &str, detail: &str) { + let (rect, _) = ui.allocate_exact_size(vec2(10.0, 10.0), Sense::hover()); ui.painter() .circle_filled(rect.center(), 4.0, status.color()); + let mut tooltip = status.message(device); + if !detail.is_empty() { + tooltip += &format!("\n{detail}"); + } + ui.label(RichText::new(status.short(device)).color(theme::TEXT_WEAK)) + .on_hover_text(tooltip); } /// Wait between attempts to open the camera or the mic @@ -99,19 +111,48 @@ const NOISE_REMOVAL_HELP: &str = "Subtracts an estimate of the uncorrelated nois const EQUAL_BINS_HELP: &str = "Gives every frequency bin the same weight. Otherwise the low \ bins, where speech has most of its energy but the array has almost no resolution, dominate \ and the zones get wide. Bins more than 20 dB below the strongest are left out."; +const CLEAN_EQUAL_BINS_HELP: &str = "Gives every frequency bin with a source the same weight \ + in the CLEAN-SC map. Otherwise the loudest bins decide, and those are the low ones, where \ + the array cannot separate a source from its reflection off a wall or the ceiling."; +const SKIP_TRANSIENTS_HELP: &str = "Leaves impulsive sounds (snapping, clapping, a hammer) out \ + of the map entirely. Off, they are shown like any other source, but can never be speech: \ + a hand snapping next to a talker is inside the beam toward the talker, both carry the same \ + audio, and only a map without the snaps tells them apart."; const BAND_HELP: &str = "Frequency range summed into the map. Lower frequencies give wider \ zones (the array is only 12.6 cm wide); above about 4 kHz ghost sources (grating lobes) can \ appear. For a single tone, narrow the band around it."; const OVERLAY_HELP: &str = "Dynamic range of the overlay: everything more than this many dB \ below the loudest point is transparent. Smaller values show only the strongest sources."; -/// Radius of the dot per estimated source in screen points, and size of its level label -const PEAK_RADIUS: f32 = 7.0; -const PEAK_FONT_SIZE: f32 = 14.0; +/// Track marker radius and label size as a fraction of the drawn image height (scales with the +/// screen), with lower limits in screen points +const MARKER_RADIUS: f32 = 0.011; +const MARKER_RADIUS_MIN: f32 = 8.0; +const MARKER_FONT: f32 = 0.02; +const MARKER_FONT_MIN: f32 = 15.0; + +/// Space between the status indicators, and the slider width in the settings menu +const GAP: f32 = 12.0; +const MENU_SLIDER_WIDTH: f32 = 160.0; +/// Band slider: rail width, handle radius and the text to its right +const BAND_RAIL_WIDTH: f32 = 200.0; +const BAND_TEXT_WIDTH: f32 = 90.0; +const HANDLE_RADIUS: f32 = 6.0; +const SETTINGS_MENU_WIDTH: f32 = 340.0; +/// Status and settings in the top-right corner: distance from the window edge, opacity while the +/// pointer is elsewhere, and how close it must come to show them fully +const CONTROLS_OFFSET: f32 = 20.0; +const CONTROLS_IDLE_OPACITY: f32 = 0.3; +const CONTROLS_HOVER_MARGIN: f32 = 40.0; /// Slider limits const NU_RANGE: RangeInclusive = 1.0..=100.0; const CLEAN_MAX_ITER_RANGE: RangeInclusive = 1..=30; -const DB_RANGE_RANGE: RangeInclusive = 1.0..=20.0; +const DB_RANGE_RANGE: RangeInclusive = 0.1..=20.0; +const SOURCE_RANGE: RangeInclusive = 0.0..=15.0; +const COHERENCE_RANGE: RangeInclusive = -20.0..=0.0; +/// Speech threshold limits +const SPEECH_THRESHOLD_MIN: f32 = 0.3; +const SPEECH_THRESHOLD_MAX: f32 = 0.99; /// Band limits in Hz; above about 6 kHz grating lobes appear in the image const F_MIN_HZ: f64 = 100.0; const F_MAX_HZ: f64 = 6000.0; @@ -130,6 +171,10 @@ struct MapData { /// [width, height] in grid pixels size: [usize; 2], power: Vec, + /// Tracks of this map, without their audio + tracks: Vec, + /// Speech state per track id; empty without speech detection + speech: HashMap, } pub struct App { @@ -141,20 +186,38 @@ pub struct App { last_map: Option, /// Overlay range in dB below the map maximum; applies immediately db_range: f64, - /// Peak search; `enabled` is toggled in the GUI + /// Peak search of the pipeline, for the tooltips; `enabled` (show the dots), + /// `min_above_floor_db` and `min_coherence_db` (which tracks are shown) are edited in the + /// GUI and apply immediately peaks: PeakConfig, - /// Peaks of `last_map` - found_peaks: Vec, + /// Tracking of the pipeline, for the tooltip + tracking: TrackerConfig, + /// Speech detection on (from `speech.enabled`, toggled in the GUI); the threshold slider is + /// shown only then + speech_enabled: bool, + /// Speech threshold edited in the GUI + speech_threshold: f32, + /// Read by the audio thread once per map (the threshold as f32 bits); no pipeline restart + shared_speech_enabled: Arc, + shared_speech_threshold: Arc, /// Settings edited in the GUI settings: PipelineSettings, /// Settings read by the audio thread; updated from `settings` when no slider is dragged shared_settings: Arc>, - /// Camera selection, status and list, shared with the camera thread + /// Camera selection, status, list and measured format, shared with the camera thread camera: Arc, /// `camera.name` from the config, shown for `CameraChoice::Auto` camera_name: String, /// Written by the audio thread audio_status: Arc>, + /// The settings of `sound2view.toml`; only what differs from them is saved + base: GuiState, + /// Last written to `GUI_STATE_PATH` (the differences from `base`) + saved: GuiState, + /// Screen area of the status and settings controls in the previous frame, for the fade + controls_rect: Rect, + /// Settings menu open in the previous frame + menu_open: bool, } impl eframe::App for App { @@ -180,40 +243,26 @@ impl eframe::App for App { let camera_status = self.camera.status.lock().unwrap().clone(); let audio_status = self.audio_status.lock().unwrap().clone(); + redraw_map |= self.controls_overlay(ui, &camera_status, &audio_status); - let top_frame = egui::Frame::new() - .fill(theme::PANEL) - .inner_margin(Margin::symmetric(16, 10)); - egui::Panel::top("controls") - .frame(top_frame) - .show(ui, |ui| { - ui.horizontal(|ui| { - ui.label(RichText::new("sound2view").strong().size(18.0)); - ui.add_space(24.0); - self.camera_control(ui); - ui.add_space(16.0); - redraw_map |= self.controls(ui); - - ui.with_layout(Layout::right_to_left(Align::Center), |ui| { - status_indicator(ui, &camera_status, "Camera"); - ui.add_space(12.0); - status_indicator(ui, &audio_status, "Mic"); - }); - }); - }); - - // Apply pipeline settings only when no slider is dragged: each change restarts the - // pipeline, and the sliding covariance needs a few frames for the first map + // Apply and save the settings only when no slider is dragged: each pipeline change + // restarts the pipeline, and the sliding covariance needs a few frames for the first map let pointer_down = ui.input(|i| i.pointer.any_down()); if !pointer_down { - let mut shared = self.shared_settings.lock().unwrap(); - if *shared != self.settings { - *shared = self.settings.clone(); + { + let mut shared = self.shared_settings.lock().unwrap(); + if *shared != self.settings { + *shared = self.settings.clone(); + } + } + let changes = self.gui_state().changes_from(&self.base); + if changes != self.saved { + changes.save(); + self.saved = changes; } } if redraw_map && let Some(map) = &self.last_map { - self.found_peaks = find_peaks(&map.power, map.size, &self.peaks); let rgba = render::map_rgba(&map.power, self.db_range); let image = ColorImage::from_rgba_unmultiplied(map.size, &rgba); // Linear filtering interpolates the coarse grid smoothly to the image size @@ -264,7 +313,16 @@ impl eframe::App for App { if self.peaks.enabled && let Some(map) = &self.last_map { - draw_peaks(painter, rect, map.size, &self.found_peaks); + let hover = ui.input(|input| input.pointer.hover_pos()); + draw_tracks( + painter, + rect, + map.size, + &map.tracks, + &map.speech, + hover, + &self.peaks, + ); } } @@ -284,7 +342,14 @@ impl eframe::App for App { } impl App { - pub fn new(cc: &eframe::CreationContext, config: Config) -> Self { + /// `config` with the saved GUI settings already applied, `base` the settings of + /// `sound2view.toml` alone, `camera` the saved camera name (`None`: automatic) + pub fn new( + cc: &eframe::CreationContext, + config: Config, + base: GuiState, + camera: Option, + ) -> Self { theme::apply(&cc.egui_ctx); let settings = PipelineSettings { @@ -293,13 +358,17 @@ impl App { }; let shared_settings = Arc::new(Mutex::new(settings.clone())); let camera = Arc::new(CameraShared { - choice: Mutex::new(CameraChoice::Auto), + choice: Mutex::new(camera.map_or(CameraChoice::Auto, CameraChoice::Name)), status: Mutex::new(DeviceStatus::Opening), cameras: Mutex::new(Vec::new()), + measured: Mutex::new(String::new()), }); let audio_status = Arc::new(Mutex::new(DeviceStatus::Opening)); + let shared_speech_enabled = Arc::new(AtomicBool::new(config.speech.enabled)); + let shared_speech_threshold = + Arc::new(AtomicU32::new(config.speech.on_threshold.to_bits())); - Self { + let mut app = Self { rx: spawn_camera( cc.egui_ctx.clone(), config.camera.clone(), @@ -311,68 +380,169 @@ impl App { config.clone(), Arc::clone(&shared_settings), Arc::clone(&audio_status), + Arc::clone(&shared_speech_enabled), + Arc::clone(&shared_speech_threshold), ), map_texture: None, last_map: None, db_range: config.display.db_range, peaks: config.peaks.clone(), - found_peaks: Vec::new(), + tracking: config.tracking.clone(), + speech_enabled: config.speech.enabled, + speech_threshold: config.speech.on_threshold, + shared_speech_enabled, + shared_speech_threshold, settings, shared_settings, camera, camera_name: config.camera.name.clone(), audio_status, - } + base, + saved: GuiState::default(), + controls_rect: Rect::NOTHING, + menu_open: false, + }; + // The file as loaded; written again only after a change + app.saved = app.gui_state().changes_from(&app.base); + app } - /// Camera dropdown: "Auto" (first camera matching `camera.name`) or one listed camera. - /// The field of view in the config belongs to the configured camera, so the overlay only - /// fits other cameras with the same field of view. - fn camera_control(&self, ui: &mut egui::Ui) { - let cameras = self.camera.cameras.lock().unwrap().clone(); - let current = self.camera.choice.lock().unwrap().clone(); - let auto_label = format!("Auto ({})", self.camera_name); - let entry_label = |entry: &CameraEntry| format!("{} ({})", entry.name, entry.index); + /// All settings menu values + fn gui_state(&self) -> GuiState { + let camera = match &*self.camera.choice.lock().unwrap() { + CameraChoice::Auto => None, + CameraChoice::Name(name) => Some(name.clone()), + }; + GuiState { + beamformer: Some(self.settings.beamformer.clone()), + f_lo_hz: Some(self.settings.pipeline.f_lo_hz), + f_hi_hz: Some(self.settings.pipeline.f_hi_hz), + skip_transients: Some(self.settings.pipeline.skip_transients), + db_range: Some(self.db_range), + tracks: Some(self.peaks.enabled), + min_above_floor_db: Some(self.peaks.min_above_floor_db), + min_coherence_db: Some(self.peaks.min_coherence_db), + speech: Some(self.speech_enabled), + speech_threshold: Some(self.speech_threshold), + camera, + } + } - let selected_text = match ¤t { - CameraChoice::Auto => auto_label.clone(), - CameraChoice::Index(index) => cameras - .iter() - .find(|entry| entry.index == *index) - .map_or_else(|| format!("Camera {index}"), entry_label), + /// Mic and camera status and the settings button in the top-right corner, over the image. + /// Faint until the pointer comes near or the menu is open, so they hide little of the + /// picture. Returns true if the overlay needs a redraw. + fn controls_overlay( + &mut self, + ui: &mut egui::Ui, + camera_status: &DeviceStatus, + audio_status: &DeviceStatus, + ) -> bool { + let ctx = ui.ctx().clone(); + let near = ctx.pointer_hover_pos().is_some_and(|pos| { + self.controls_rect + .expand(CONTROLS_HOVER_MARGIN) + .contains(pos) + }); + let opacity = if near || self.menu_open { + 1.0 + } else { + CONTROLS_IDLE_OPACITY }; - let mut choice = current.clone(); - label(ui, "Camera").on_hover_text(format!( - "Auto opens the first camera whose name contains \"{}\" (camera.name). The overlay \ - only lines up with cameras that match the configured field of view or calibration.", - self.camera_name - )); - egui::ComboBox::from_id_salt("camera") - .selected_text(selected_text) - .width(CAMERA_COMBO_WIDTH) - .show_ui(ui, |ui| { - ui.selectable_value(&mut choice, CameraChoice::Auto, auto_label); - for entry in &cameras { - let label = entry_label(entry); - ui.selectable_value(&mut choice, CameraChoice::Index(entry.index), label); - } + let mut redraw_map = false; + let response = egui::Area::new(egui::Id::unique("controls")) + .anchor(Align2::RIGHT_TOP, vec2(-CONTROLS_OFFSET, CONTROLS_OFFSET)) + .show(&ctx, |ui| { + ui.set_opacity(opacity); + egui::Frame::new() + .fill(theme::PANEL) + .corner_radius(8) + .inner_margin(Margin::symmetric(12, 8)) + .show(ui, |ui| { + ui.horizontal(|ui| { + status_indicator(ui, audio_status, "Mic", ""); + ui.add_space(GAP); + let measured = self.camera.measured.lock().unwrap().clone(); + status_indicator(ui, camera_status, "Camera", &measured); + ui.add_space(GAP); + redraw_map = self.settings_menu(ui); + }); + }); }); + self.controls_rect = response.response.rect; + redraw_map + } - if choice != current { - *self.camera.choice.lock().unwrap() = choice; - set_status(ui.ctx(), &self.camera.status, DeviceStatus::Opening); - } + /// The settings button and its menu with all settings; returns true if the overlay needs + /// a redraw + fn settings_menu(&mut self, ui: &mut egui::Ui) -> bool { + let config = MenuConfig::new().close_behavior(PopupCloseBehavior::CloseOnClickOutside); + let (_, inner) = MenuButton::new("\u{2699} Settings") + .config(config) + .ui(ui, |ui| { + ui.set_min_width(SETTINGS_MENU_WIDTH); + ui.spacing_mut().slider_width = MENU_SLIDER_WIDTH; + self.menu_contents(ui) + }); + self.menu_open = inner.is_some(); + inner.is_some_and(|inner| inner.inner) } - /// Beamformer and its parameters, band, overlay range and peak toggle; returns true if the - /// overlay needs a redraw - fn controls(&mut self, ui: &mut egui::Ui) -> bool { - let beamformer = &mut self.settings.beamformer; + /// All settings, in sections; returns true if the overlay needs a redraw + fn menu_contents(&mut self, ui: &mut egui::Ui) -> bool { + heading(ui, "Beamformer"); + self.beamformer_controls(ui); + + ui.separator(); + heading(ui, "Map"); let pipeline = &mut self.settings.pipeline; + band_slider(ui, &mut pipeline.f_lo_hz, &mut pipeline.f_hi_hz).on_hover_text(BAND_HELP); + let redraw_map = ui + .horizontal(|ui| { + label(ui, "Overlay").on_hover_text(OVERLAY_HELP); + ui.add( + Slider::new(&mut self.db_range, DB_RANGE_RANGE) + .logarithmic(true) + .suffix(" dB") + .max_decimals(1), + ) + .on_hover_text(OVERLAY_HELP) + .changed() + }) + .inner; + ui.checkbox( + &mut self.settings.pipeline.skip_transients, + "Skip transients", + ) + .on_hover_text(SKIP_TRANSIENTS_HELP); + ui.separator(); + heading(ui, "Sources"); + self.source_controls(ui); + + ui.separator(); + heading(ui, "Speech"); + self.speech_controls(ui); + + ui.separator(); + heading(ui, "Camera"); + self.camera_control(ui); + + ui.separator(); + ui.label( + RichText::new(format!( + "Changes are saved to {GUI_STATE_PATH}; delete it to use sound2view.toml again." + )) + .small() + .color(theme::TEXT_WEAK), + ); + redraw_map + } + + /// Method, its main parameter and its options + fn beamformer_controls(&mut self, ui: &mut egui::Ui) { + let beamformer = &mut self.settings.beamformer; let method_help = beamformer.method.description(); - label(ui, "Beamformer").on_hover_text(method_help); egui::ComboBox::from_id_salt("method") .selected_text(beamformer.method.label()) .show_ui(ui, |ui| { @@ -384,109 +554,336 @@ impl App { .response .on_hover_text(method_help); - // Parameter of the selected method match beamformer.method { Method::Das => {} Method::Functional => { - ui.add_space(16.0); - label(ui, "nu").on_hover_text(NU_HELP); - ui.add( - Slider::new(&mut beamformer.nu, NU_RANGE) - .logarithmic(true) - .max_decimals(0), - ) - .on_hover_text(NU_HELP); + ui.horizontal(|ui| { + label(ui, "nu").on_hover_text(NU_HELP); + ui.add( + Slider::new(&mut beamformer.nu, NU_RANGE) + .logarithmic(true) + .max_decimals(0), + ) + .on_hover_text(NU_HELP); + }); ui.checkbox(&mut beamformer.functional_noise_removal, "Noise removal") .on_hover_text(NOISE_REMOVAL_HELP); ui.checkbox(&mut beamformer.functional_equal_bins, "Equal bins") .on_hover_text(EQUAL_BINS_HELP); } Method::CleanSc => { - ui.add_space(16.0); let sources_help = format!( "At most this many sources are removed per frequency bin. More finds weaker \ sources, but also more reflections. Bins whose strongest point is more than \ {} dB below the strongest bin add nothing.", beamformer.clean_stop_db ); - label(ui, "Sources").on_hover_text(&sources_help); - ui.add(Slider::new( - &mut beamformer.clean_max_iter, - CLEAN_MAX_ITER_RANGE, - )) - .on_hover_text(sources_help); + ui.horizontal(|ui| { + label(ui, "Sources").on_hover_text(&sources_help); + ui.add(Slider::new( + &mut beamformer.clean_max_iter, + CLEAN_MAX_ITER_RANGE, + )) + .on_hover_text(&sources_help); + }); + ui.checkbox(&mut beamformer.clean_equal_bins, "Equal bins") + .on_hover_text(CLEAN_EQUAL_BINS_HELP); } } + } - ui.add_space(16.0); - label(ui, "Band").on_hover_text(BAND_HELP); - // Each end is limited by the other, so the band never collapses - ui.add( - Slider::new( - &mut pipeline.f_lo_hz, - F_MIN_HZ..=pipeline.f_hi_hz - MIN_BAND_HZ, - ) - .logarithmic(true) - .suffix(" Hz") - .max_decimals(0), - ) - .on_hover_text(BAND_HELP); - ui.add( - Slider::new( - &mut pipeline.f_hi_hz, - pipeline.f_lo_hz + MIN_BAND_HZ..=F_MAX_HZ, + /// Track dots and the thresholds that decide which tracks are shown + fn source_controls(&mut self, ui: &mut egui::Ui) { + ui.checkbox(&mut self.peaks.enabled, "Show sources") + .on_hover_text(format!( + "Dots at the tracked sources: local maxima of the map within {} dB of the \ + loudest point, at least {} grid cells apart and not on the image border \ + (sources outside the image end up there), at most {}, central ones first. A dot \ + appears once its maximum was found in {} of {} maps within {} grid cells, and \ + only if the source is loud and coherent enough (the two sliders below). Green: \ + the audio from its direction is classified as speech. Hover over a dot for \ + its values.", + self.peaks.range_db, + self.peaks.min_dist_cells, + self.peaks.max_count, + self.tracking.min_hits, + self.tracking.confirm_window, + self.tracking.max_dist_cells, + )); + if !self.peaks.enabled { + return; + } + + let floor_help = format!( + "How much louder (coherent power from its direction) a source must be than the \ + quietest moment of the last minute to get a dot. Toward the image border it needs \ + up to {} dB more (the array resolves best in the center). Lower it if a talker \ + gets no dot, raise it if dots appear on nothing.", + self.peaks.center_weight_db + ); + ui.horizontal(|ui| { + label(ui, "Above floor").on_hover_text(&floor_help); + ui.add( + Slider::new(&mut self.peaks.min_above_floor_db, SOURCE_RANGE) + .suffix(" dB") + .max_decimals(0), ) - .logarithmic(true) - .suffix(" Hz") - .max_decimals(0), - ) - .on_hover_text(BAND_HELP); + .on_hover_text(&floor_help); + }); - ui.add_space(16.0); - label(ui, "Overlay").on_hover_text(OVERLAY_HELP); - let db_range_changed = ui - .add( - Slider::new(&mut self.db_range, DB_RANGE_RANGE) + let coherence_help = "How much of the sound above 2 kHz must arrive coherently from the \ + source's direction. A point source or a snap reaches about 0 dB, diffuse room noise \ + and mic noise stay far below, whatever their level. Raise it if dots appear on noise, \ + lower it if real sources get none."; + ui.horizontal(|ui| { + label(ui, "Coherence").on_hover_text(coherence_help); + ui.add( + Slider::new(&mut self.peaks.min_coherence_db, COHERENCE_RANGE) .suffix(" dB") - .max_decimals(1), + .max_decimals(0), ) - .on_hover_text(OVERLAY_HELP) - .changed(); + .on_hover_text(coherence_help); + }); + } - ui.add_space(16.0); - ui.checkbox(&mut self.peaks.enabled, "Peaks") + /// Speech classification on/off and its threshold (no pipeline restart) + fn speech_controls(&mut self, ui: &mut egui::Ui) { + let speech_help = "Classify the audio from each track as speech (green dot). Off: no \ + classifier runs."; + if ui + .checkbox(&mut self.speech_enabled, "Detect speech") + .on_hover_text(speech_help) + .changed() + { + self.shared_speech_enabled + .store(self.speech_enabled, Ordering::Relaxed); + } + if !self.speech_enabled { + return; + } + let help = "Speech probability (smoothed) at which a track turns green. It stays green \ + for speech.hold_ms after the last time the probability was this high. Raise it if \ + noise or music turns green, lower it if speech stays blue."; + ui.horizontal(|ui| { + label(ui, "Threshold").on_hover_text(help); + let changed = ui + .add( + Slider::new( + &mut self.speech_threshold, + SPEECH_THRESHOLD_MIN..=SPEECH_THRESHOLD_MAX, + ) + .max_decimals(2), + ) + .on_hover_text(help) + .changed(); + if changed { + self.shared_speech_threshold + .store(self.speech_threshold.to_bits(), Ordering::Relaxed); + } + }); + } + + /// Camera dropdown: "Auto" (first camera matching `camera.name`) or one listed camera, by + /// name (survives replugging), plus the measured format. The field of view in the config + /// belongs to the configured camera, so the overlay only fits other cameras with the same + /// field of view. + fn camera_control(&self, ui: &mut egui::Ui) { + let mut names: Vec = self + .camera + .cameras + .lock() + .unwrap() + .iter() + .map(|entry| entry.name.clone()) + .collect(); + // One entry per name: a camera can show up several times (V4L2 metadata node) + names.dedup(); + let current = self.camera.choice.lock().unwrap().clone(); + let auto_label = format!("Auto ({})", self.camera_name); + let selected_text = match ¤t { + CameraChoice::Auto => auto_label.clone(), + CameraChoice::Name(name) => name.clone(), + }; + + let mut choice = current.clone(); + egui::ComboBox::from_id_salt("camera") + .selected_text(selected_text) + .width(CAMERA_COMBO_WIDTH) + .show_ui(ui, |ui| { + ui.selectable_value(&mut choice, CameraChoice::Auto, auto_label); + for name in names { + let label = name.clone(); + ui.selectable_value(&mut choice, CameraChoice::Name(name), label); + } + }) + .response .on_hover_text(format!( - "Blue dots at the estimated source positions: local maxima of the map within {} \ - dB of the loudest point, at least {} grid cells apart, at most {}. The label is \ - the level relative to the loudest point.", - self.peaks.range_db, self.peaks.min_dist_cells, self.peaks.max_count + "Auto opens the first camera whose name contains \"{}\" (camera.name). The \ + overlay only lines up with cameras that match the configured field of view or \ + calibration.", + self.camera_name )); - db_range_changed + + let measured = self.camera.measured.lock().unwrap().clone(); + if !measured.is_empty() { + label(ui, &measured); + } + + if choice != current { + *self.camera.choice.lock().unwrap() = choice; + set_status(ui.ctx(), &self.camera.status, DeviceStatus::Opening); + } } } -/// Dot with level label per peak; `rect` is where the map is drawn, `size` its grid size -fn draw_peaks(painter: &egui::Painter, rect: Rect, [width, height]: [usize; 2], peaks: &[Peak]) { - let font = FontId::proportional(PEAK_FONT_SIZE); - for peak in peaks { +/// Dot with id and level per shown track (see `Track::shown`), with its values while the pointer +/// (`hover`) is on the dot; `rect` is where the map is drawn, `size` its grid size +fn draw_tracks( + painter: &egui::Painter, + rect: Rect, + [width, height]: [usize; 2], + tracks: &[Track], + speech: &HashMap, + hover: Option, + peaks: &PeakConfig, +) { + let radius = (MARKER_RADIUS * rect.height()).max(MARKER_RADIUS_MIN); + let font = FontId::proportional((MARKER_FONT * rect.height()).max(MARKER_FONT_MIN)); + let shade = Color32::from_black_alpha(170); + let shown = tracks.iter().filter(|track| track.shown(peaks)); + for track in shown { // Grid cell centers at (col + 0.5) / width of the image - let x = rect.left() + (peak.col as f32 + 0.5) / width as f32 * rect.width(); - let y = rect.top() + (peak.row as f32 + 0.5) / height as f32 * rect.height(); + let x = rect.left() + (track.col as f32 + 0.5) / width as f32 * rect.width(); + let y = rect.top() + (track.row as f32 + 0.5) / height as f32 * rect.height(); let center = pos2(x, y); + let status = speech.get(&track.id); + let is_speech = status.is_some_and(|s| s.speech); + + // Speech: large filled green dot with a halo; other sources: hollow blue ring. Shape and + // fill differ, not only the color. + let marker_radius = if is_speech { + painter.circle_stroke( + center, + 1.9 * radius, + Stroke::new(0.25 * radius, theme::SPEECH.gamma_multiply(0.5)), + ); + painter.circle( + center, + 1.3 * radius, + theme::SPEECH, + Stroke::new(2.0, Color32::WHITE), + ); + 1.9 * radius + } else { + painter.circle_filled(center, radius, shade); + painter.circle_stroke(center, radius, Stroke::new(0.35 * radius, theme::PEAK)); + radius + }; + + let mut text = format!("#{} {:.1} dB", track.id, track.level_db); + if let Some(status) = status { + text += &format!(" {:.0} %", 100.0 * status.probability); + } + if is_speech { + text += " speech"; + } else if let Some(source) = status.and_then(|s| s.echo_of) { + text += &format!(" echo of #{source}"); + } + if hover.is_some_and(|pos| pos.distance(center) <= 2.0 * radius) { + text += &format!( + " | {:+.0} dB above the noise floor (needs {:.0}), {:.0} dB coherent above \ + 2 kHz{}", + track.source_db, + peaks.min_above_floor_db + track.edge_penalty_db, + track.source_upper_db, + if track.steady { "" } else { ", impulsive" } + ); + } + // Label on a pill: green with dark text for speech, dark with white text otherwise + let (pill, text_color) = if is_speech { + (theme::SPEECH, theme::BACKGROUND) + } else { + (shade, Color32::WHITE) + }; + let galley = painter.layout_no_wrap(text, font.clone(), text_color); + let padding = egui::vec2(0.4, 0.15) * font.size; + let min = pos2( + x + marker_radius + 0.4 * font.size, + y - galley.size().y / 2.0 - padding.y, + ); + let pill_rect = Rect::from_min_size(min, galley.size() + 2.0 * padding); + painter.rect_filled(pill_rect, 0.3 * font.size, pill); + painter.galley(min + padding, galley, text_color); + } +} + +/// Both band limits on one slider with two handles (log scale from `F_MIN_HZ` to `F_MAX_HZ`), +/// the values to its right. A drag or click moves the nearer handle; each end stays +/// `MIN_BAND_HZ` from the other. +fn band_slider(ui: &mut egui::Ui, f_lo: &mut f64, f_hi: &mut f64) -> egui::Response { + let size = vec2( + BAND_RAIL_WIDTH + BAND_TEXT_WIDTH, + ui.spacing().interact_size.y, + ); + let (rect, response) = ui.allocate_exact_size(size, Sense::click_and_drag()); + let rail = Rect::from_x_y_ranges( + rect.left() + HANDLE_RADIUS..=rect.left() + BAND_RAIL_WIDTH - HANDLE_RADIUS, + rect.center().y - 2.0..=rect.center().y + 2.0, + ); + let log_span = (F_MAX_HZ / F_MIN_HZ).ln(); + let to_x = |f: f64| rail.left() + ((f / F_MIN_HZ).ln() / log_span) as f32 * rail.width(); + let to_f = |x: f32| { + let t = ((x - rail.left()) / rail.width()).clamp(0.0, 1.0) as f64; + F_MIN_HZ * (t * log_span).exp() + }; + + // The handle nearer to the pointer when the interaction starts is the one that moves + let handle_id = response.id.with("upper handle"); + if let Some(pos) = response.interact_pointer_pos() { + if response.drag_started() || response.clicked() { + let upper = (pos.x - to_x(*f_hi)).abs() < (pos.x - to_x(*f_lo)).abs(); + ui.data_mut(|data| data.insert_temp(handle_id, upper)); + } + if response.dragged() || response.clicked() { + let upper = ui + .data(|data| data.get_temp::(handle_id)) + .unwrap_or(false); + let f = to_f(pos.x); + if upper { + *f_hi = f.clamp(*f_lo + MIN_BAND_HZ, F_MAX_HZ); + } else { + *f_lo = f.clamp(F_MIN_HZ, *f_hi - MIN_BAND_HZ); + } + } + } + + let painter = ui.painter(); + painter.rect_filled(rail, 2.0, ui.visuals().widgets.inactive.bg_fill); + let (x_lo, x_hi) = (to_x(*f_lo), to_x(*f_hi)); + let selected = Rect::from_x_y_ranges(x_lo..=x_hi, rail.y_range()); + painter.rect_filled(selected, 2.0, theme::ACCENT); + for x in [x_lo, x_hi] { + let center = pos2(x, rail.center().y); painter.circle( center, - PEAK_RADIUS, - theme::PEAK, - Stroke::new(2.0, Color32::WHITE), - ); - painter.text( - pos2(x + PEAK_RADIUS + 4.0, y), - Align2::LEFT_CENTER, - format!("{:.1} dB", peak.level_db), - font.clone(), + HANDLE_RADIUS, Color32::WHITE, + Stroke::new(1.0, theme::ACCENT), ); } + painter.text( + pos2(rect.left() + BAND_RAIL_WIDTH + 6.0, rect.center().y), + Align2::LEFT_CENTER, + format!("{f_lo:.0}-{f_hi:.0} Hz"), + FontId::proportional(14.0), + theme::TEXT, + ); + response +} + +/// Section title in the settings menu +fn heading(ui: &mut egui::Ui, text: &str) { + ui.label(RichText::new(text).strong()); } /// Muted label in front of a control @@ -505,8 +902,8 @@ fn set_status(ctx: &egui::Context, status: &Mutex, new: DeviceStat enum CameraChoice { /// First camera whose name contains `camera.name` from the config Auto, - /// Camera with this index, see `CameraEntry` - Index(u32), + /// Camera with this name (chosen in the GUI; the name survives replugging, the index not) + Name(String), } /// Camera state shared between the GUI and the camera thread @@ -517,6 +914,8 @@ struct CameraShared { status: Mutex, /// Cameras for the dropdown; refreshed by the thread on every open attempt cameras: Mutex>, + /// Delivered frame size and rate, e.g. "1920x1080, 30 fps"; updated about once a second + measured: Mutex, } /// Width of the camera dropdown; camera names are long @@ -547,7 +946,7 @@ fn spawn_camera( let (width, height, fps) = (config.width, config.height, config.fps); let camera = match &choice { CameraChoice::Auto => CameraSource::open_by_name(&config.name, width, height, fps), - CameraChoice::Index(index) => CameraSource::new(*index, width, height, fps), + CameraChoice::Name(name) => CameraSource::open_by_name(name, width, height, fps), }; let mut camera = match camera { Ok(camera) => { @@ -564,6 +963,7 @@ fn spawn_camera( let mut failed = 0; let mut last_error = String::new(); let mut size_checked = false; + let mut rate = FrameRate::new(); while failed < CAMERA_MAX_FAILED_FRAMES { // Another camera selected: drop this one (end of the loop body), then open it if *shared.choice.lock().unwrap() != choice { @@ -578,6 +978,10 @@ fn spawn_camera( } }; failed = 0; + if let Some(fps) = rate.frame() { + *shared.measured.lock().unwrap() = + format!("{}x{}, {fps:.0} fps", frame.width, frame.height); + } // The overlay is stretched over the frame; intrinsics only fit the configured size if !size_checked { size_checked = true; @@ -604,6 +1008,36 @@ fn spawn_camera( rx } +/// Frames per second, measured over about `FRAME_RATE_WINDOW` +struct FrameRate { + start: Instant, + frames: u32, +} + +/// Interval of the measured frame rate +const FRAME_RATE_WINDOW: Duration = Duration::from_secs(1); + +impl FrameRate { + fn new() -> Self { + Self { + start: Instant::now(), + frames: 0, + } + } + + /// Count a frame; the rate once per window, else `None` + fn frame(&mut self) -> Option { + self.frames += 1; + let elapsed = self.start.elapsed(); + if elapsed < FRAME_RATE_WINDOW { + return None; + } + let fps = f64::from(self.frames) / elapsed.as_secs_f64(); + *self = Self::new(); + Some(fps) + } +} + /// Sleep up to `timeout`, but return as soon as the GUI selects another camera than `current` fn wait_for_other_choice(choice: &Mutex, current: &CameraChoice, timeout: Duration) { let mut waited = Duration::ZERO; @@ -625,11 +1059,17 @@ fn spawn_audio( mut config: Config, settings: Arc>, status: Arc>, + speech_enabled: Arc, + speech_threshold: Arc, ) -> Receiver { let (tx, rx) = sync_channel(1); thread::spawn(move || { let sample_rate = config.audio.sample_rate as f64; + // Loaded when first enabled and kept (reset per pipeline run, since track ids start + // over); stays `None` after a failed load + let mut speech: Option = None; + let mut speech_failed = false; loop { let mut mic = @@ -649,26 +1089,61 @@ fn spawn_audio( config.beamformer = current.beamformer.clone(); config.pipeline = current.pipeline.clone(); let mut gui_closed = false; + if let Some(detector) = &mut speech { + detector.reset(); + } - let on_map = - |power: &[f64], layout: &MapLayout| -> Result, BoxError> { - let map = MapData { - size: [layout.grid_width, layout.grid_height], - power: power.to_vec(), - }; - match tx.try_send(map) { - Ok(()) => ctx.request_repaint(), - Err(TrySendError::Full(_)) => {} - Err(TrySendError::Disconnected(_)) => { - gui_closed = true; - return Ok(ControlFlow::Break(())); - } + let on_map = |power: &[f64], + layout: &MapLayout, + tracks: &[Track]| + -> Result, BoxError> { + let enabled = speech_enabled.load(Ordering::Relaxed); + if enabled && speech.is_none() && !speech_failed { + speech = SpeechDetector::load(&config.speech) + .inspect_err(|e| eprintln!("No speech detection: {e}")) + .ok(); + speech_failed = speech.is_none(); + } + let speech = match &mut speech { + Some(detector) if enabled => { + let threshold = speech_threshold.load(Ordering::Relaxed); + detector.set_on_threshold(f32::from_bits(threshold)); + detector.update(tracks)?; + detector.statuses() } - if *settings.lock().unwrap() != current { - return Ok(ControlFlow::Break(())); + // Off: forget the tracks, so that turning it on starts fresh + Some(detector) => { + detector.reset(); + HashMap::new() } - Ok(ControlFlow::Continue(())) + None => HashMap::new(), }; + let tracks = tracks + .iter() + .map(|track| Track { + samples: Vec::new(), + ..track.clone() + }) + .collect(); + let map = MapData { + size: [layout.grid_width, layout.grid_height], + power: power.to_vec(), + tracks, + speech, + }; + match tx.try_send(map) { + Ok(()) => ctx.request_repaint(), + Err(TrySendError::Full(_)) => {} + Err(TrySendError::Disconnected(_)) => { + gui_closed = true; + return Ok(ControlFlow::Break(())); + } + } + if *settings.lock().unwrap() != current { + return Ok(ControlFlow::Break(())); + } + Ok(ControlFlow::Continue(())) + }; let result = run_configured(&mut mic, sample_rate, &config, on_map); diff --git a/crates/app/src/gui_state.rs b/crates/app/src/gui_state.rs new file mode 100644 index 0000000..77d7f8b --- /dev/null +++ b/crates/app/src/gui_state.rs @@ -0,0 +1,171 @@ +//! GUI settings kept between runs in `sound2view-gui.toml` (working directory): everything that +//! can be changed in the settings menu. Written whenever a setting changes and applied on top of +//! `sound2view.toml` at the next start; delete the file to go back to `sound2view.toml`. + +use std::fs; + +use serde::{Deserialize, Serialize}; + +use crate::config::{BeamformerConfig, Config}; + +pub const GUI_STATE_PATH: &str = "sound2view-gui.toml"; + +/// Settings menu values; `None` (not in the file) keeps the value from `sound2view.toml` +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default)] +pub struct GuiState { + pub beamformer: Option, + pub f_lo_hz: Option, + pub f_hi_hz: Option, + pub skip_transients: Option, + pub db_range: Option, + pub tracks: Option, + pub min_above_floor_db: Option, + pub min_coherence_db: Option, + pub speech: Option, + pub speech_threshold: Option, + /// Name of the chosen camera; not set: automatic (`camera.name` from the config) + pub camera: Option, +} + +impl GuiState { + /// All values of `config` (camera automatic) + pub fn of_config(config: &Config) -> Self { + Self { + beamformer: Some(config.beamformer.clone()), + f_lo_hz: Some(config.pipeline.f_lo_hz), + f_hi_hz: Some(config.pipeline.f_hi_hz), + skip_transients: Some(config.pipeline.skip_transients), + db_range: Some(config.display.db_range), + tracks: Some(config.peaks.enabled), + min_above_floor_db: Some(config.peaks.min_above_floor_db), + min_coherence_db: Some(config.peaks.min_coherence_db), + speech: Some(config.speech.enabled), + speech_threshold: Some(config.speech.on_threshold), + camera: None, + } + } + + /// Only the values that differ from `base`. Saving only those keeps later edits of + /// `sound2view.toml` in effect for everything not changed in the GUI. + pub fn changes_from(&self, base: &Self) -> Self { + fn changed(value: &Option, base: &Option) -> Option { + (value != base).then(|| value.clone()).flatten() + } + Self { + beamformer: changed(&self.beamformer, &base.beamformer), + f_lo_hz: changed(&self.f_lo_hz, &base.f_lo_hz), + f_hi_hz: changed(&self.f_hi_hz, &base.f_hi_hz), + skip_transients: changed(&self.skip_transients, &base.skip_transients), + db_range: changed(&self.db_range, &base.db_range), + tracks: changed(&self.tracks, &base.tracks), + min_above_floor_db: changed(&self.min_above_floor_db, &base.min_above_floor_db), + min_coherence_db: changed(&self.min_coherence_db, &base.min_coherence_db), + speech: changed(&self.speech, &base.speech), + speech_threshold: changed(&self.speech_threshold, &base.speech_threshold), + camera: changed(&self.camera, &base.camera), + } + } + + /// Read `GUI_STATE_PATH`; empty if it does not exist, with a message if it cannot be read + pub fn load() -> Self { + let Ok(text) = fs::read_to_string(GUI_STATE_PATH) else { + return Self::default(); + }; + toml::from_str(&text).unwrap_or_else(|e| { + eprintln!("{GUI_STATE_PATH} ignored: {e}"); + Self::default() + }) + } + + /// Write to `GUI_STATE_PATH`; an error is printed, the GUI goes on + pub fn save(&self) { + let text = match toml::to_string(self) { + Ok(text) => text, + Err(e) => { + eprintln!("cannot save GUI settings: {e}"); + return; + } + }; + let header = "# Written by the sound2view GUI; delete to use sound2view.toml again\n\n"; + if let Err(e) = fs::write(GUI_STATE_PATH, format!("{header}{text}")) { + eprintln!("cannot write {GUI_STATE_PATH}: {e}"); + } + } + + /// `config` with the saved values; unchanged if the result would be invalid + pub fn apply(&self, config: &Config) -> Config { + let mut applied = config.clone(); + if let Some(beamformer) = &self.beamformer { + applied.beamformer = beamformer.clone(); + } + let pipeline = &mut applied.pipeline; + set(&mut pipeline.f_lo_hz, self.f_lo_hz); + set(&mut pipeline.f_hi_hz, self.f_hi_hz); + set(&mut pipeline.skip_transients, self.skip_transients); + set(&mut applied.display.db_range, self.db_range); + set(&mut applied.peaks.enabled, self.tracks); + set( + &mut applied.peaks.min_above_floor_db, + self.min_above_floor_db, + ); + set(&mut applied.peaks.min_coherence_db, self.min_coherence_db); + set(&mut applied.speech.enabled, self.speech); + set(&mut applied.speech.on_threshold, self.speech_threshold); + + match applied.validate() { + Ok(()) => applied, + Err(e) => { + eprintln!("{GUI_STATE_PATH} ignored: {e}"); + config.clone() + } + } + } +} + +fn set(target: &mut T, value: Option) { + if let Some(value) = value { + *target = value; + } +} + +#[cfg(test)] +mod test { + use super::GuiState; + use crate::config::Config; + + /// Only changed values are kept, and applying them gives the changed config back. + #[test] + fn changes_round_trip() { + let config = Config::default(); + let base = GuiState::of_config(&config); + let mut current = base.clone(); + current.db_range = Some(5.0); + current.camera = Some("USB Camera".to_string()); + + let changes = current.changes_from(&base); + assert_eq!(changes.db_range, Some(5.0)); + assert_eq!(changes.camera.as_deref(), Some("USB Camera")); + assert_eq!(changes.f_lo_hz, None); + assert_eq!(changes.beamformer, None); + + let text = toml::to_string(&changes).unwrap(); + let loaded: GuiState = toml::from_str(&text).unwrap(); + assert_eq!(loaded, changes); + assert_eq!(loaded.apply(&config).display.db_range, 5.0); + } + + /// Saved values that make the config invalid are ignored as a whole. + #[test] + fn invalid_values_ignored() { + let config = Config::default(); + let state = GuiState { + db_range: Some(-1.0), + f_lo_hz: Some(1000.0), + ..GuiState::default() + }; + let applied = state.apply(&config); + assert_eq!(applied.display.db_range, config.display.db_range); + assert_eq!(applied.pipeline.f_lo_hz, config.pipeline.f_lo_hz); + } +} diff --git a/crates/app/src/main.rs b/crates/app/src/main.rs index f28e7cc..b6313e8 100644 --- a/crates/app/src/main.rs +++ b/crates/app/src/main.rs @@ -1,7 +1,9 @@ use std::{ + collections::{HashMap, HashSet, hash_map::Entry}, env, error::Error, - fs, + fs::{self, File}, + io::BufWriter, ops::ControlFlow, thread, time::{Duration, Instant}, @@ -10,11 +12,12 @@ use std::{ use image::RgbaImage; use sound2view_core::{ BoxError, FrameSource, ImageSource, Point3, + dsp::beam_signal::TRACK_SAMPLE_RATE, dsp::beamformer::{ Normalization, clean_sc::CleanSc, delay_and_sum::Das, functional::Functional, }, geometry::mic_array::MicArray, - peaks::{PeakConfig, find_peaks}, + peaks::PeakConfig, pipeline::{MapLayout, Pipeline, PipelineError, run}, render, source::{ @@ -22,24 +25,35 @@ use sound2view_core::{ simulator::{Simulator, scene::Scene}, video::{self, CameraSource}, }, + tracking::Track, }; -use crate::config::{CONFIG_PATH, Config, Method}; +use crate::{ + config::{CONFIG_PATH, Config, Method}, + gui_state::GuiState, + speech::{SpeechConfig, SpeechDetector}, +}; mod config; mod gui; +mod gui_state; +mod speech; mod theme; -/// Number of maps written in mic mode +/// Number of maps written in mic mode, unless given as argument const MIC_MAPS: usize = 20; /// Frames timed in camshot mode const CAMSHOT_FRAMES: usize = 60; +/// Cores the beamformers (rayon) leave free in the GUI: for the camera thread (frame copy) and +/// drawing. With all cores busy computing maps the camera image stuttered on a Mac mini. +const GUI_FREE_CORES: usize = 2; /// CLI modes wait this long for the user to grant camera access (macOS asks on the first run) const CAMERA_AUTH_TIMEOUT: Duration = Duration::from_secs(10); /// Usage (settings from `sound2view.toml`, see `config.rs`): /// - `sound2view [scene.toml]`: simulate a scene, defaults to `config.toml` -/// - `sound2view mic`: capture from the UMA-16 +/// - `sound2view mic [maps]`: capture from the UMA-16, `MIC_MAPS` maps unless given (about 9 per +/// second) /// - `sound2view probe`: list audio input devices /// - `sound2view camprobe`: list cameras and their formats /// - `sound2view camshot`: time camera frames and save one as `camshot.png` @@ -64,7 +78,15 @@ fn main() -> Result<(), Box> { video::initialize(); run_gui(Config::load(CONFIG_PATH)?) } - "mic" => run_mic(&Config::load(CONFIG_PATH)?), + "mic" => { + let maps = match env::args().nth(2) { + Some(n) => n + .parse() + .map_err(|_| format!("mic: not a map count: {n}"))?, + None => MIC_MAPS, + }; + run_mic(&Config::load(CONFIG_PATH)?, maps) + } path => run_sim(path, &Config::load(CONFIG_PATH)?), } } @@ -113,12 +135,13 @@ fn run_sim(path: &str, config: &Config) -> Result<(), Box> { let sample_rate = scene.sample_rate; let mut sim = Simulator::new(scene, MicArray::uma16())?; - let on_map = save_maps(src_pos, None, config.peaks.clone()); + let speech = load_speech(&config.speech); + let on_map = save_maps(src_pos, None, config.peaks.clone(), speech); run_configured(&mut sim, sample_rate, config, on_map)?; Ok(()) } -fn run_mic(config: &Config) -> Result<(), Box> { +fn run_mic(config: &Config, maps: usize) -> Result<(), Box> { let audio = &config.audio; println!( "Capturing from {} at {} Hz", @@ -131,7 +154,12 @@ fn run_mic(config: &Config) -> Result<(), Box> { &mut mic, sample_rate, config, - save_maps(Vec::new(), Some(MIC_MAPS), config.peaks.clone()), + save_maps( + Vec::new(), + Some(maps), + config.peaks.clone(), + load_speech(&config.speech), + ), )?; println!("dropped {} chunks", mic.dropped()); @@ -145,63 +173,107 @@ fn run_configured>( source: &mut S, sample_rate: f64, config: &Config, - on_map: impl FnMut(&[f64], &MapLayout) -> Result, BoxError>, + on_map: impl FnMut(&[f64], &MapLayout, &[Track]) -> Result, BoxError>, ) -> Result<(), PipelineError> { let camera = config.camera.camera(); let pipeline = &config.pipeline; let b = &config.beamformer; + let (peaks, tracking) = (&config.peaks, &config.tracking); match b.method { Method::Das => { - let pipeline = Pipeline::new(sample_rate, pipeline, camera, |steering| { - Das::new(steering, true, Normalization::TrueLocation) - })?; + let pipeline = + Pipeline::new(sample_rate, pipeline, peaks, tracking, camera, |steering| { + Das::new(steering, true, Normalization::TrueLocation) + })?; run(source, pipeline, on_map) } Method::Functional => { - let pipeline = Pipeline::new(sample_rate, pipeline, camera, |steering| { - Functional::new( - steering, - b.nu, - b.functional_noise_removal, - b.functional_equal_bins, - ) - })?; + let pipeline = + Pipeline::new(sample_rate, pipeline, peaks, tracking, camera, |steering| { + Functional::new( + steering, + b.nu, + b.functional_noise_removal, + b.functional_equal_bins, + ) + })?; run(source, pipeline, on_map) } Method::CleanSc => { - let pipeline = Pipeline::new(sample_rate, pipeline, camera, |steering| { - CleanSc::new( - steering, - b.clean_gain, - b.clean_max_iter, - b.clean_stop_db, - b.clean_beam_cells, - true, - ) - })?; + let pipeline = + Pipeline::new(sample_rate, pipeline, peaks, tracking, camera, |steering| { + CleanSc::new( + steering, + b.clean_gain, + b.clean_max_iter, + b.clean_stop_db, + b.clean_beam_cells, + true, + b.clean_equal_bins, + ) + })?; run(source, pipeline, on_map) } } } +/// Speech detector from `config`; `None` if disabled or the model does not load (with a note) +fn load_speech(config: &SpeechConfig) -> Option { + if !config.enabled { + return None; + } + SpeechDetector::load(config) + .inspect_err(|e| eprintln!("No speech detection: {e}")) + .ok() +} + /// Map handler for `run`: writes each map as `map_NNN.png` with a red cross per `src_pos` and a -/// blue dot per peak (if `peaks.enabled`). +/// dot per shown track (confirmed and `peaks.min_above_floor_db` above the noise floor, if +/// `peaks.enabled`; green if classified as speech by `speech`), and the audio of each shown +/// track to `track_NNN.wav` (see [`TrackRecorder`]). Prints when a shown track starts or stops +/// being speech. /// /// Stops after `max_maps` maps; `None` runs until the stream ends. fn save_maps( src_pos: Vec, max_maps: Option, peaks: PeakConfig, -) -> impl FnMut(&[f64], &MapLayout) -> Result, BoxError> { + mut speech: Option, +) -> impl FnMut(&[f64], &MapLayout, &[Track]) -> Result, BoxError> { let mut i = 0; - move |power, layout| { - let found = if peaks.enabled { - find_peaks(power, [layout.grid_width, layout.grid_height], &peaks) - } else { - Vec::new() - }; - render::map_image(power, layout, &src_pos, &found).save(format!("map_{i:03}.png"))?; + let mut recorder = TrackRecorder::default(); + let mut talking: HashSet = HashSet::new(); + move |power, layout, tracks| { + let shown: Vec = tracks + .iter() + .filter(|track| track.shown(&peaks)) + .cloned() + .collect(); + let mut speech_ids = Vec::new(); + if let Some(detector) = &mut speech { + detector.update(tracks)?; + for track in &shown { + let is_speech = detector.status(track.id).is_some_and(|s| s.speech); + if is_speech { + speech_ids.push(track.id); + } + if is_speech != talking.contains(&track.id) { + let change = if is_speech { "speech" } else { "no speech" }; + println!("map {i}: track #{} {change}", track.id); + if is_speech { + talking.insert(track.id); + } else { + talking.remove(&track.id); + } + } + } + } + + let drawn = if peaks.enabled { &shown[..] } else { &[] }; + render::map_image(power, layout, &src_pos, drawn, &speech_ids) + .save(format!("map_{i:03}.png"))?; + recorder.record(&shown)?; i += 1; if max_maps.is_some_and(|max| i >= max) { Ok(ControlFlow::Break(())) @@ -211,14 +283,70 @@ fn save_maps( } } +/// Writes the audio of each confirmed track to `track_NNN.wav` (NNN = track id), mono 32-bit +/// float at `TRACK_SAMPLE_RATE`, to check by ear whether a beam isolates its source. Audio from +/// before the confirmation is not written. The files are finalized when the recorder is dropped. +#[derive(Default)] +struct TrackRecorder { + writers: HashMap>>, +} + +impl TrackRecorder { + fn record(&mut self, tracks: &[Track]) -> Result<(), BoxError> { + let spec = hound::WavSpec { + channels: 1, + sample_rate: TRACK_SAMPLE_RATE as u32, + bits_per_sample: 32, + sample_format: hound::SampleFormat::Float, + }; + for track in tracks.iter().filter(|track| track.confirmed) { + let writer = match self.writers.entry(track.id) { + Entry::Occupied(entry) => entry.into_mut(), + Entry::Vacant(entry) => { + let path = format!("track_{:03}.wav", track.id); + entry.insert(hound::WavWriter::create(path, spec)?) + } + }; + for &sample in &track.samples { + writer.write_sample(sample)?; + } + } + Ok(()) + } +} + +/// Window and dock icon, rendered from `assets/logo.svg` +const ICON_PNG: &[u8] = include_bytes!("../../../assets/logo-256.png"); + fn run_gui(config: Config) -> Result<(), Box> { + leave_cores_for_gui(); + // Settings menu values of the last run on top of sound2view.toml; the GUI saves only what + // differs from `base` + let base = GuiState::of_config(&config); + let saved = GuiState::load(); + let config = saved.apply(&config); + let icon = eframe::icon_data::from_png_bytes(ICON_PNG)?; eframe::run_native( "sound2view", eframe::NativeOptions { - viewport: eframe::egui::ViewportBuilder::default().with_maximized(true), + viewport: eframe::egui::ViewportBuilder::default() + .with_maximized(true) + .with_icon(icon), ..Default::default() }, - Box::new(|cc| Ok(Box::new(gui::App::new(cc, config)))), + Box::new(move |cc| Ok(Box::new(gui::App::new(cc, config, base, saved.camera)))), )?; Ok(()) } + +/// Limit rayon's global pool to all cores but `GUI_FREE_CORES` (at least one thread) +fn leave_cores_for_gui() { + let cores = thread::available_parallelism().map_or(1, |n| n.get()); + let threads = cores.saturating_sub(GUI_FREE_CORES).max(1); + if let Err(e) = rayon::ThreadPoolBuilder::new() + .num_threads(threads) + .build_global() + { + eprintln!("rayon thread pool: {e}"); + } +} diff --git a/crates/app/src/speech.rs b/crates/app/src/speech.rs new file mode 100644 index 0000000..7fe933c --- /dev/null +++ b/crates/app/src/speech.rs @@ -0,0 +1,796 @@ +//! Speech detection per tracked source: the audio of each track goes through the +//! voice-activity-detection classifier, smoothed with hysteresis per track. + +use std::collections::{HashMap, VecDeque}; + +use serde::Deserialize; +use sound2view_core::{BoxError, tracking::Track}; +use voice_activity_detection::{ + model::Model, + preprocessing::{HOP, Preprocessor, SAMPLE_RATE}, +}; + +#[derive(Clone, Debug, PartialEq, Deserialize)] +#[serde(default, deny_unknown_fields)] +pub struct SpeechConfig { + /// Classify the tracks at all + pub enabled: bool, + /// Model exported by voice-activity-detection, relative to the working directory + pub model: String, + /// Smoothed probability at which a source becomes speech, in (0, 1] + pub on_threshold: f32, + /// Speech is held this long after the last frame at `on_threshold` or above, in ms: pauses + /// between words stay speech, other sounds after the speech (clapping, snapping) end it + pub hold_ms: f32, + /// Time constant of the probability smoothing, in ms (0: no smoothing) + pub smoothing_ms: f32, + /// The smoothed probability must stay at `on_threshold` this long before speech starts, in + /// ms: transients (snapping, clapping) are shorter than a syllable + pub min_onset_ms: f32, + /// A speech track whose audio correlates at least this much (within +-20 ms) with a speech + /// track that has `echo_margin_db` more speech energy repeats that track's speech (reflection, + /// beamformer ghost, leakage into another sound) and is not speech, in [0, 1]; 1 or more + /// turns the check off + pub echo_correlation: f64, + /// See `echo_correlation`, in dB + pub echo_margin_db: f32, + /// Speech ends at once when the track moves farther than this from where it last was at + /// `on_threshold`, in grid cells: another source nearby (snapping next to the speaker) that + /// the tracker continues as the same track does not inherit the speech flag + pub max_shift_cells: f64, + /// Gain applied to the beam signal before the classifier, in dB. The classifier is trained on + /// microphone levels (speech around -25 dBFS); the UMA-16 gives speech at 1 to 2 m around + /// -60 dBFS. + pub gain_db: f32, + /// Classifier frames quieter than this (after `gain_db`) count as non-speech, in dBFS + pub min_level_db: f32, + /// A track is speech only while its share of the power received above 2 kHz that the other + /// tracks do not explain (`Track::source_upper_db`) is at least this, in dB. Below about + /// 1.5 kHz, where most speech energy is, the array has no directivity and every beam + /// carries every talker, so the classifier alone cannot tell whose speech a beam carries; + /// above 2 kHz a talker elsewhere leaks in at the sidelobe level only. + pub min_source_db: f64, +} + +impl Default for SpeechConfig { + fn default() -> Self { + Self { + enabled: true, + model: "model/speech_detector.json".to_string(), + on_threshold: 0.95, + hold_ms: 500.0, + smoothing_ms: 50.0, + min_onset_ms: 80.0, + echo_correlation: 0.2, + echo_margin_db: 6.0, + max_shift_cells: 3.0, + gain_db: 36.0, + min_level_db: -55.0, + min_source_db: -6.0, + } + } +} + +impl SpeechConfig { + pub fn validate(&self) -> Result<(), String> { + if !(self.on_threshold > 0.0 && self.on_threshold <= 1.0) { + return Err("speech.on_threshold must be in (0, 1]".to_string()); + } + if !self.gain_db.is_finite() { + return Err("speech.gain_db must be finite".to_string()); + } + let non_negative = [self.hold_ms, self.smoothing_ms, self.min_onset_ms]; + if !(non_negative.iter().all(|&v| v >= 0.0) && self.max_shift_cells >= 0.0) { + return Err( + "speech: hold_ms, smoothing_ms, min_onset_ms and max_shift_cells must be >= 0" + .to_string(), + ); + } + if self.echo_correlation.is_nan() || self.echo_correlation < 0.0 { + return Err("speech.echo_correlation must be >= 0".to_string()); + } + if !self.echo_margin_db.is_finite() || self.min_source_db.is_nan() { + return Err( + "speech: echo_margin_db must be finite, min_source_db a number".to_string(), + ); + } + Ok(()) + } + + /// Hold in classifier frames + fn hold_frames(&self) -> usize { + (self.hold_ms / FRAME_MS).round() as usize + } + + /// Minimum onset in classifier frames + fn onset_frames(&self) -> usize { + (self.min_onset_ms / FRAME_MS).round() as usize + } + + /// Weight of a new frame in the smoothed probability, 1 - exp(-frame / tau) + fn smoothing_weight(&self) -> f32 { + if self.smoothing_ms > 0.0 { + 1.0 - (-FRAME_MS / self.smoothing_ms).exp() + } else { + 1.0 + } + } +} + +/// Length of one classifier frame (`HOP` samples) in ms +const FRAME_MS: f32 = 1000.0 * HOP as f32 / SAMPLE_RATE as f32; + +/// Audio per track kept for the echo check, in samples (0.3 s) +const ECHO_WINDOW: usize = SAMPLE_RATE * 3 / 10; +/// Largest delay between a track and its echo, in samples (20 ms, about 7 m of extra path) +const ECHO_MAX_LAG: usize = SAMPLE_RATE / 50; + +/// Speech state of one track, for display +#[derive(Clone, Copy, Debug, Default, PartialEq)] +pub struct SpeechStatus { + pub speech: bool, + /// Smoothed speech probability in [0, 1] + pub probability: f32, + /// Speech-like, but a reflection or ghost of this louder track, see + /// [`SpeechConfig::echo_correlation`] + pub echo_of: Option, +} + +/// Smoothing and hysteresis of the classifier output: speech starts when the smoothed +/// probability has been at `on_threshold` for `onset_frames` and ends `hold_frames` after the +/// last frame that reached it. Only such frames re-arm the hold, so a sound the classifier finds +/// half speech-like (clapping after speech, same track) cannot keep it on. Speech also ends when +/// the track moves more than `max_shift_cells` from where it last reached `on_threshold` +/// (another source took it over). +#[derive(Default)] +struct Hysteresis { + status: SpeechStatus, + /// Frames left until speech ends + hold: usize, + /// Track position (col, row) at the last frame at `on_threshold` + anchor: [f64; 2], + /// Consecutive frames with the smoothed probability at `on_threshold` + above: usize, +} + +/// Per-frame constants derived from [`SpeechConfig`] +struct HysteresisParams { + on_threshold: f32, + min_level_db: f32, + hold_frames: usize, + onset_frames: usize, + weight: f32, + max_shift_cells: f64, +} + +impl HysteresisParams { + fn new(config: &SpeechConfig) -> Self { + Self { + on_threshold: config.on_threshold, + min_level_db: config.min_level_db, + hold_frames: config.hold_frames(), + onset_frames: config.onset_frames(), + weight: config.smoothing_weight(), + max_shift_cells: config.max_shift_cells, + } + } +} + +impl Hysteresis { + /// One classifier frame; `pos` is the track position (col, row) in grid cells + fn update(&mut self, probability: f32, rms: f32, pos: [f64; 2], params: &HysteresisParams) { + let level_db = 20.0 * rms.max(1e-5).log10(); + let probability = if level_db < params.min_level_db { + 0.0 + } else { + probability.clamp(0.0, 1.0) + }; + let status = &mut self.status; + status.probability += params.weight * (probability - status.probability); + if status.probability >= params.on_threshold { + self.above += 1; + } else { + self.above = 0; + } + + // Starts on the smoothed probability after the onset; the frame's own probability must + // reach the threshold too, both to start and to re-arm the hold and the anchor, so the + // smoothing tail of the speech can neither anchor nor restart it on a source that took + // over the track + if probability >= params.on_threshold + && (status.speech || self.above >= params.onset_frames.max(1)) + { + status.speech = true; + self.hold = params.hold_frames; + self.anchor = pos; + } else if status.speech { + let shift = (pos[0] - self.anchor[0]).hypot(pos[1] - self.anchor[1]); + self.hold = if shift > params.max_shift_cells { + 0 + } else { + self.hold.saturating_sub(1) + }; + status.speech = self.hold > 0; + } + } +} + +struct TrackSpeech { + preprocessor: Preprocessor, + hysteresis: Hysteresis, + /// The last `ECHO_WINDOW` samples, without gain + recent: Vec, + /// (probability, rms^2) of the classifier frames in the last `ECHO_WINDOW` samples, newest + /// last + frames: VecDeque<(f32, f32)>, +} + +impl TrackSpeech { + /// Energy of the speech in the echo window: frame power weighted by its speech probability. + /// Snaps and other sounds count little (low probability), leaked or reflected speech is + /// weaker than the speaker's own. + fn speech_energy(&self) -> f32 { + self.frames.iter().map(|&(p, power)| p * power).sum() + } +} + +/// Classifies the audio of every track; state per track id +pub struct SpeechDetector { + model: Model, + config: SpeechConfig, + tracks: HashMap, +} + +impl SpeechDetector { + /// Load the model from `config.model` + pub fn load(config: &SpeechConfig) -> Result { + let model = Model::load(&config.model).map_err(|e| format!("{}: {e}", config.model))?; + Ok(Self { + model, + config: config.clone(), + tracks: HashMap::new(), + }) + } + + /// Forget all tracks; call when a new pipeline run starts (track ids start over) + pub fn reset(&mut self) { + self.tracks.clear(); + } + + /// Change `on_threshold` while running; applies from the next classifier frame. In (0, 1] + /// (see [`SpeechConfig::validate`]). + pub fn set_on_threshold(&mut self, threshold: f32) { + debug_assert!(threshold > 0.0 && threshold <= 1.0); + self.config.on_threshold = threshold; + } + + /// Classify the new audio of each track (`Track::samples`, 16 kHz); tracks that are gone are + /// forgotten. + pub fn update(&mut self, tracks: &[Track]) -> Result<(), BoxError> { + self.tracks + .retain(|id, _| tracks.iter().any(|track| track.id == *id)); + let params = HysteresisParams::new(&self.config); + let gain = 10f32.powf(self.config.gain_db / 20.0); + let mut samples = Vec::new(); + + for track in tracks { + // An impulsive source (snapping next to the talker: its beam carries the talker's + // speech) or leaked speech from a talker elsewhere: not speech, whatever the + // classifier says + let present = track.steady && track.source_upper_db >= self.config.min_source_db; + let state = self.tracks.entry(track.id).or_insert_with(|| TrackSpeech { + preprocessor: Preprocessor::new(), + hysteresis: Hysteresis::default(), + recent: Vec::new(), + frames: VecDeque::new(), + }); + state.recent.extend_from_slice(&track.samples); + let excess = state.recent.len().saturating_sub(ECHO_WINDOW); + state.recent.drain(..excess); + + samples.clear(); + samples.extend(track.samples.iter().map(|s| s * gain)); + let frames = state.preprocessor.push(&samples); + if frames.is_empty() { + continue; + } + let rms: Vec = frames.iter().map(|frame| frame.rms).collect(); + let features = frames.into_iter().map(|frame| frame.features).collect(); + let probabilities = self.model.predict(features)?; + for (probability, rms) in probabilities.into_iter().zip(rms) { + let probability = if present { probability } else { 0.0 }; + state + .hysteresis + .update(probability, rms, [track.col, track.row], ¶ms); + state + .frames + .push_back((probability.clamp(0.0, 1.0), rms * rms)); + } + let excess = state.frames.len().saturating_sub(ECHO_WINDOW / HOP); + state.frames.drain(..excess); + } + + // Speech tracks that only echo a louder speech track are not speech. Decided on the + // speech flags before any is cleared, so the order of the tracks does not matter. + let speech: Vec = tracks + .iter() + .map(|track| self.tracks[&track.id].hysteresis.status.speech) + .collect(); + let echoes: Vec> = (0..tracks.len()) + .map(|i| self.echo_source(i, tracks, &speech)) + .collect(); + for (track, echo_of) in tracks.iter().zip(echoes) { + let hysteresis = &mut self.tracks.get_mut(&track.id).unwrap().hysteresis; + hysteresis.status.echo_of = echo_of; + if echo_of.is_some() { + hysteresis.status.speech = false; + hysteresis.hold = 0; + } + } + Ok(()) + } + + /// The track whose speech `tracks[i]` repeats (reflection, beamformer ghost, or another + /// sound with the speech leaking in): another speech track with at least `echo_margin_db` + /// more speech energy (see [`TrackSpeech::speech_energy`]) whose audio correlates at least + /// `echo_correlation`. `None` if `tracks[i]` is not speech. + /// + /// Below about 1.5 kHz, where most speech energy is, the 12.6 cm array has almost no + /// directivity, so every beam carries a good part of every talker: a snapping track next to + /// the speaker is speech-like through the leaked speech and correlates only about 0.35 with + /// the speaker (the snaps dilute it). Its speech energy is 9 to 18 dB below the speaker's. + /// Not the louder track: the snapping is louder in the map and in the beam signal. Two + /// talkers within `echo_margin_db` of each other both stay speech. + fn echo_source(&self, i: usize, tracks: &[Track], speech: &[bool]) -> Option { + let track = &tracks[i]; + let state = &self.tracks[&track.id]; + if !speech[i] || self.config.echo_correlation >= 1.0 || state.recent.len() < ECHO_WINDOW / 2 + { + return None; + } + let margin = 10f32.powf(self.config.echo_margin_db / 10.0); + let energy = state.speech_energy(); + tracks + .iter() + .zip(speech) + .filter(|&(other, &other_speech)| { + let other_state = &self.tracks[&other.id]; + other.id != track.id + && other_speech + && other_state.recent.len() >= ECHO_WINDOW / 2 + && other_state.speech_energy() >= margin * energy + }) + .map(|(other, _)| { + let other_recent = &self.tracks[&other.id].recent; + ( + other.id, + max_correlation(&state.recent, other_recent, ECHO_MAX_LAG), + ) + }) + .filter(|&(_, correlation)| correlation >= self.config.echo_correlation) + .max_by(|a, b| a.1.total_cmp(&b.1)) + .map(|(id, _)| id) + } + + /// Current state of track `id`, `None` before its first classifier frame + pub fn status(&self, id: u64) -> Option { + self.tracks.get(&id).map(|state| state.hysteresis.status) + } + + /// State of every current track + pub fn statuses(&self) -> HashMap { + self.tracks + .iter() + .map(|(&id, state)| (id, state.hysteresis.status)) + .collect() + } +} + +fn mean_square(x: &[f32]) -> f64 { + x.iter().map(|&v| f64::from(v).powi(2)).sum::() / x.len().max(1) as f64 +} + +/// Largest normalized cross-correlation |sum a[n] b[n + lag]| / sqrt(sum a^2 sum b^2) over +/// |lag| <= `max_lag`; `a` and `b` end at the same time, the longer one is cut at the start +fn max_correlation(a: &[f32], b: &[f32], max_lag: usize) -> f64 { + let len = a.len().min(b.len()); + let (a, b) = (&a[a.len() - len..], &b[b.len() - len..]); + let norm = (mean_square(a) * mean_square(b)).sqrt() * len as f64; + if norm == 0.0 { + return 0.0; + } + let max_lag = max_lag.min(len.saturating_sub(1)); + let dot = |x: &[f32], y: &[f32]| -> f64 { + x.iter() + .zip(y) + .map(|(&x, &y)| f64::from(x) * f64::from(y)) + .sum() + }; + (0..=max_lag) + .flat_map(|lag| { + [ + dot(&a[..len - lag], &b[lag..]), + dot(&a[lag..], &b[..len - lag]), + ] + }) + .map(f64::abs) + .fold(0.0, f64::max) + / norm +} + +#[cfg(test)] +mod test { + use sound2view_core::tracking::Track; + + use super::{ + FRAME_MS, Hysteresis, HysteresisParams, SpeechConfig, SpeechDetector, max_correlation, + }; + + fn config() -> SpeechConfig { + SpeechConfig { + model: concat!( + env!("CARGO_MANIFEST_DIR"), + "/../../model/speech_detector.json" + ) + .to_string(), + // The test signals are at microphone level already + gain_db: 0.0, + ..SpeechConfig::default() + } + } + + /// Track 0 with `samples` + fn track(samples: &[f32]) -> Track { + track_with_id(0, 0.0, samples) + } + + /// Track `id` with map level `level_db` + fn track_with_id(id: u64, level_db: f64, samples: &[f32]) -> Track { + Track { + id, + col: 0.0, + row: 0.0, + level_db, + source_db: 0.0, + source_upper_db: 0.0, + edge_penalty_db: 0.0, + steady: true, + hits: 1, + recent: 1, + missed: 0, + confirmed: true, + samples: samples.to_vec(), + } + } + + /// Feed `audio` in pieces of one map (about 0.11 s at 16 kHz), return the speech flag after + /// each piece + fn speech_per_map(audio: &[f32]) -> Vec { + let mut detector = SpeechDetector::load(&config()).unwrap(); + audio + .chunks(1707) + .map(|chunk| { + detector.update(&[track(chunk)]).unwrap(); + detector.status(0).is_some_and(|status| status.speech) + }) + .collect() + } + + fn speech_fixture() -> Vec { + let path = concat!(env!("CARGO_MANIFEST_DIR"), "/testdata/speech_16k.wav"); + let mut reader = hound::WavReader::open(path).unwrap(); + assert_eq!(reader.spec().sample_rate, 16_000); + reader + .samples::() + .map(|s| s.unwrap() as f32 / 32768.0) + .collect() + } + + /// Defaults with `on_threshold` 0.7: the hysteresis tests use made-up probabilities around it + fn hysteresis_config() -> SpeechConfig { + SpeechConfig { + on_threshold: 0.7, + ..SpeechConfig::default() + } + } + + /// White noise, uniform in [-amplitude, amplitude] (xorshift) + fn noise(len: usize, seed: u32, amplitude: f32) -> Vec { + let mut x = seed; + (0..len) + .map(|_| { + x ^= x << 13; + x ^= x >> 17; + x ^= x << 5; + (x as f32 / u32::MAX as f32 - 0.5) * 2.0 * amplitude + }) + .collect() + } + + /// Probability `p` at speech level for `ms` milliseconds; returns the speech flag at the end + fn feed(state: &mut Hysteresis, params: &HysteresisParams, p: f32, ms: f32) -> bool { + feed_at(state, params, p, ms, [10.0, 10.0]) + } + + /// As `feed`, with the track at `pos` + fn feed_at( + state: &mut Hysteresis, + params: &HysteresisParams, + p: f32, + ms: f32, + pos: [f64; 2], + ) -> bool { + for _ in 0..(ms / FRAME_MS).round() as usize { + state.update(p, 0.1, pos, params); + } + state.status.speech + } + + /// Speech holds through short pauses and a drop to 0.2, ends `hold_ms` after the last frame + /// at `on_threshold`, also when a half speech-like sound follows; no speech below the level + /// gate. + #[test] + fn hysteresis_hold_and_level_gate() { + let params = HysteresisParams::new(&hysteresis_config()); + assert_eq!(params.hold_frames, 50); + + let mut state = Hysteresis::default(); + assert!( + !feed(&mut state, ¶ms, 0.6, 1000.0), + "below on_threshold" + ); + assert!(feed(&mut state, ¶ms, 1.0, 500.0)); + assert!(feed(&mut state, ¶ms, 0.2, 400.0), "held at 0.2"); + // Syllables and pauses + for _ in 0..10 { + assert!(feed(&mut state, ¶ms, 0.9, 150.0)); + assert!(feed(&mut state, ¶ms, 0.05, 250.0)); + } + assert!(feed(&mut state, ¶ms, 0.9, 150.0)); + assert!( + feed(&mut state, ¶ms, 0.0, 400.0), + "held during the pause" + ); + assert!( + !feed(&mut state, ¶ms, 0.0, 200.0), + "ended after hold_ms" + ); + + // Clapping after speech: about 0.45, never on_threshold + assert!(feed(&mut state, ¶ms, 1.0, 500.0)); + assert!( + feed(&mut state, ¶ms, 0.45, 400.0), + "held after the speech" + ); + assert!( + !feed(&mut state, ¶ms, 0.45, 200.0), + "a 0.45 sound does not keep it" + ); + + let mut quiet = Hysteresis::default(); + for _ in 0..50 { + quiet.update(1.0, 1e-4, [10.0, 10.0], ¶ms); + } + assert!(!quiet.status.speech, "-80 dB is below the level gate"); + } + + /// Another source taking over the track ends speech at once; the speaker moving a little + /// does not. + #[test] + fn hysteresis_ends_when_track_moves() { + let params = HysteresisParams::new(&hysteresis_config()); + let mut state = Hysteresis::default(); + assert!(feed_at(&mut state, ¶ms, 1.0, 500.0, [10.0, 10.0])); + assert!( + feed_at(&mut state, ¶ms, 0.3, 200.0, [11.5, 11.0]), + "within 3 cells" + ); + // Speech at the new position re-anchors + assert!(feed_at(&mut state, ¶ms, 1.0, 200.0, [12.0, 11.0])); + assert!( + !feed_at(&mut state, ¶ms, 0.3, 20.0, [16.0, 11.0]), + "snapping 4 cells away" + ); + } + + /// Bursts shorter than the onset (snapping, clapping) do not start speech, a syllable does. + #[test] + fn short_bursts_do_not_start_speech() { + let params = HysteresisParams::new(&hysteresis_config()); + let mut state = Hysteresis::default(); + for _ in 0..10 { + assert!(!feed(&mut state, ¶ms, 1.0, 60.0)); + assert!(!feed(&mut state, ¶ms, 0.0, 300.0)); + } + assert!(feed(&mut state, ¶ms, 1.0, 200.0)); + } + + /// A delayed, weaker copy correlates with the original; independent noise does not. + #[test] + fn correlation_finds_delayed_copy() { + let original = noise(4800, 1, 0.1); + let other = noise(4800, 2, 0.1); + let mut copy = vec![0.0; 120]; + copy.extend(original[..4800 - 120].iter().map(|v| 0.3 * v)); + let copy: Vec = copy + .iter() + .zip(noise(4800, 3, 0.01)) + .map(|(c, n)| c + n) + .collect(); + + let echo = max_correlation(©, &original, 320); + let independent = max_correlation(&other, &original, 320); + assert!(echo > 0.9, "echo {echo}"); + assert!(independent < 0.1, "independent {independent}"); + // Cut to the shorter one at the start + assert!(max_correlation(©[2000..], &original, 320) > 0.9); + } + + /// A reflection of a speaker (weaker, 5 ms later) is not speech, the speaker is. + #[test] + fn reflection_is_not_speech() { + let speech = speech_fixture(); + let mut reflection = vec![0.0; 80]; + reflection.extend(speech[..speech.len() - 80].iter().map(|v| 0.3 * v)); + let room = noise(reflection.len(), 5, 0.002); + let reflection: Vec = reflection.iter().zip(room).map(|(r, n)| r + n).collect(); + + let mut detector = SpeechDetector::load(&config()).unwrap(); + let (mut speaker_maps, mut echo_maps, mut maps) = (0, 0, 0); + for (a, b) in speech.chunks(1707).zip(reflection.chunks(1707)) { + // The beamformer map ranks the speaker above its reflection + detector + .update(&[track_with_id(0, 0.0, a), track_with_id(1, -6.0, b)]) + .unwrap(); + let speaker = detector.status(0).unwrap_or_default(); + let echo = detector.status(1).unwrap_or_default(); + assert!(!echo.speech, "reflection is speech in map {maps}"); + speaker_maps += usize::from(speaker.speech); + echo_maps += usize::from(echo.echo_of == Some(0)); + maps += 1; + } + assert!( + speaker_maps > maps / 2, + "speaker is speech in {speaker_maps} of {maps} maps" + ); + assert!(echo_maps > 0, "the reflection was never speech-like"); + } + + /// Snapping while talking: the snapping track is louder and shares the snaps and the speech + /// with the speaker's track (leakage both ways). The speaker stays speech, the snapping track + /// (speech-like through the leaked speech) is its echo. + #[test] + fn snapping_does_not_make_speaker_an_echo() { + let speech = speech_fixture(); + let hiss = noise(speech.len(), 9, 1.0); + // A snap every 300 ms, decaying with 8 ms + let snaps: Vec = hiss + .iter() + .enumerate() + .map(|(n, v)| 0.5 * (-((n % 4800) as f32) / 128.0).exp() * v) + .collect(); + let speaker: Vec = speech + .iter() + .zip(&snaps) + .map(|(s, k)| s + 0.3 * k) + .collect(); + let snapping: Vec = speech + .iter() + .zip(&snaps) + .map(|(s, k)| 0.1 * s + k) + .collect(); + + let mut detector = SpeechDetector::load(&config()).unwrap(); + let (mut speaker_maps, mut snapping_maps, mut maps) = (0, 0, 0); + for (a, b) in speaker.chunks(1707).zip(snapping.chunks(1707)) { + // The snapping is louder in the map + detector + .update(&[track_with_id(0, -3.0, a), track_with_id(1, 0.0, b)]) + .unwrap(); + let status = detector.status(0).unwrap_or_default(); + assert_eq!(status.echo_of, None, "speaker is an echo in map {maps}"); + speaker_maps += usize::from(status.speech); + snapping_maps += usize::from(detector.status(1).unwrap_or_default().speech); + maps += 1; + } + assert!( + speaker_maps > maps / 2, + "speaker is speech in {speaker_maps} of {maps} maps" + ); + // The speech leaking into the snapping track is an echo of the speaker + assert_eq!( + snapping_maps, 0, + "snapping is speech in {snapping_maps} of {maps} maps" + ); + } + + /// Two talkers at the same level, each leaking into the other's beam, both stay speech. + #[test] + fn two_talkers_are_both_speech() { + let speech = speech_fixture(); + // The second talker says the same text 1.3 s later (independent signal) + let offset = 20_800; + let other: Vec = speech + .iter() + .cycle() + .skip(offset) + .take(speech.len()) + .copied() + .collect(); + let a: Vec = speech + .iter() + .zip(&other) + .map(|(s, o)| s + 0.3 * o) + .collect(); + let b: Vec = other + .iter() + .zip(&speech) + .map(|(o, s)| o + 0.3 * s) + .collect(); + + let mut detector = SpeechDetector::load(&config()).unwrap(); + let mut together = [0, 0]; + for (a, b) in a.chunks(1707).zip(b.chunks(1707)) { + detector + .update(&[track_with_id(0, 0.0, a), track_with_id(1, 0.0, b)]) + .unwrap(); + for id in [0, 1] { + together[id] += usize::from(detector.status(id as u64).unwrap_or_default().speech); + } + } + // While one talker pauses, its beam carries only the other's leaked speech: that is an + // echo. Talking, each stays speech nearly as often as alone (78 % at on_threshold 0.95: + // the other's leaked speech lowers the probability a little). + for (id, alone) in [&speech, &other].into_iter().enumerate() { + let alone = speech_per_map(alone).iter().filter(|&&s| s).count(); + assert!( + together[id] * 10 >= alone * 7, + "track {id}: speech in {} maps together, {alone} alone", + together[id] + ); + } + } + + /// Speech that does not arrive from the track's direction (a talker elsewhere leaking into + /// the beam), or an impulsive track (snapping next to the talker), is not speech. + #[test] + fn leaked_speech_and_impulsive_tracks_are_not_speech() { + for (upper_db, steady) in [(-12.0, true), (0.0, false)] { + let mut detector = SpeechDetector::load(&config()).unwrap(); + for chunk in speech_fixture().chunks(1707) { + let mut track = track(chunk); + track.source_upper_db = upper_db; + track.steady = steady; + detector.update(&[track]).unwrap(); + let status = detector.status(0).unwrap_or_default(); + assert!(!status.speech && status.probability == 0.0, "{status:?}"); + } + } + } + + /// Synthesized speech (macOS `say`, 4.7 s) is detected as speech. + #[test] + fn speech_fixture_is_speech() { + let flags = speech_per_map(&speech_fixture()); + let speech = flags.iter().filter(|&&s| s).count(); + assert!( + speech > flags.len() / 2, + "speech in {speech} of {} maps", + flags.len() + ); + } + + /// White noise and a steady tone at speech level are not speech. + #[test] + fn noise_and_tone_are_not_speech() { + let noise = noise(48_000, 1, 0.1); + let tone: Vec = (0..48_000) + .map(|n| 0.1 * (2.0 * std::f32::consts::PI * 1000.0 * n as f32 / 16_000.0).sin()) + .collect(); + + for (name, audio) in [("noise", noise), ("tone", tone)] { + let flags = speech_per_map(&audio); + assert!(!flags.iter().any(|&s| s), "{name}: {flags:?}"); + } + } +} diff --git a/crates/app/src/theme.rs b/crates/app/src/theme.rs index 1c52d98..7188da0 100644 --- a/crates/app/src/theme.rs +++ b/crates/app/src/theme.rs @@ -20,6 +20,8 @@ pub const OK: Color32 = Color32::from_rgb(0x3f, 0xb9, 0x50); pub const ERROR: Color32 = Color32::from_rgb(0xf8, 0x51, 0x49); /// Estimated source positions, contrasts with the amber overlay pub const PEAK: Color32 = Color32::from_rgb(0x2f, 0x81, 0xf7); +/// Source classified as speech +pub const SPEECH: Color32 = Color32::from_rgb(0x3f, 0xb9, 0x50); /// Rounding of buttons, combo boxes and popups const RADIUS: u8 = 6; diff --git a/crates/app/testdata/speech_16k.wav b/crates/app/testdata/speech_16k.wav new file mode 100644 index 0000000..cb75098 Binary files /dev/null and b/crates/app/testdata/speech_16k.wav differ diff --git a/crates/core/src/dsp.rs b/crates/core/src/dsp.rs index e0c41c4..6e99584 100644 --- a/crates/core/src/dsp.rs +++ b/crates/core/src/dsp.rs @@ -1,5 +1,6 @@ //! Signal processing: STFT, covariance and beamforming. +pub mod beam_signal; pub mod beamformer; pub mod covariance; pub mod stft; diff --git a/crates/core/src/dsp/beam_signal.rs b/crates/core/src/dsp/beam_signal.rs new file mode 100644 index 0000000..b00e047 --- /dev/null +++ b/crates/core/src/dsp/beam_signal.rs @@ -0,0 +1,357 @@ +//! Audio from one direction: delay and sum toward a point per STFT frame, back to a time signal +//! by overlap-add, decimated to `TRACK_SAMPLE_RATE` for a speech classifier. + +use std::{f64::consts::PI, sync::Arc}; + +use nalgebra::SVector; +use realfft::{ComplexToReal, RealFftPlanner}; + +use crate::{Complex64, distance_to_delay, geometry::mic_array::MicArray}; + +/// Sample rate of the beam signal in Hz (what speech classifiers expect) +pub const TRACK_SAMPLE_RATE: f64 = 16_000.0; + +/// Integer factor from `sample_rate` down to [`TRACK_SAMPLE_RATE`], `None` if there is none +pub fn decimation_factor(sample_rate: f64) -> Option { + let factor = sample_rate / TRACK_SAMPLE_RATE; + (factor >= 1.0 && factor.fract() == 0.0).then_some(factor as usize) +} + +/// Delay and sum toward one point, as a stream of time samples. +/// +/// Per STFT frame y(f_k) = w_k^H x(f_k) with w = a e^{j omega r_ref / c} / (a^H a), so a source at +/// the point comes out as its signal s(t - r_ref / c) referenced to 1 m, with r_ref the mean +/// mic distance. The inverse FFT of y gives the windowed segment; consecutive segments overlap +/// by N/2 and the periodic Hann window sums to 1, so overlap-add reconstructs the signal. +/// +/// The weights shift each mic's frame circularly by (r_m - r_ref) / c, at most about 13 samples +/// at 48 kHz, where the window is nearly 0. Removing the whole delay r_m / c instead (about 280 +/// samples at 2 m) wraps the window around the frame and breaks the overlap-add: an error at +/// every hop, heard as a buzz. +pub struct BeamSignal { + /// w_k per bin for the current point + weights: Vec>, + /// FFT bin spacing in Hz + df: f64, + plan: Arc>, + spectrum: Vec, + segment: Vec, + scratch: Vec, + /// Overlap-add accumulator, N samples; the first N/2 are complete after each frame + ola: Vec, + /// 1 / (N * stft_scale): undoes the gain of the unnormalized inverse FFT and the STFT scale + gain: f64, + decimator: Decimator, + /// Samples at `TRACK_SAMPLE_RATE` since the last `take_output` + output: Vec, +} + +impl BeamSignal { + /// - `fft_len`, `stft_scale`: of the STFT that produces the frames, see + /// [`crate::dsp::stft::Stft::scale`] + /// - `sample_rate`: in Hz, an integer multiple of [`TRACK_SAMPLE_RATE`] + /// - `distances`: r_m from each mic to the point, see [`MicArray::get_distance_from`] + /// + /// Panics if `sample_rate` has no integer decimation factor. + pub fn new(fft_len: usize, stft_scale: f64, sample_rate: f64, distances: &[f64; M]) -> Self { + let factor = decimation_factor(sample_rate) + .expect("sample rate must be an integer multiple of TRACK_SAMPLE_RATE"); + let plan = RealFftPlanner::new().plan_fft_inverse(fft_len); + + let mut beam = Self { + weights: vec![SVector::zeros(); fft_len / 2 + 1], + df: sample_rate / fft_len as f64, + spectrum: plan.make_input_vec(), + segment: plan.make_output_vec(), + scratch: plan.make_scratch_vec(), + plan, + ola: vec![0.0; fft_len], + gain: 1.0 / (fft_len as f64 * stft_scale), + decimator: Decimator::new(factor), + output: Vec::new(), + }; + beam.steer(distances); + beam + } + + /// Point the beam at the point with mic distances `distances` + pub fn steer(&mut self, distances: &[f64; M]) { + let delay_ref = distance_to_delay(distances.iter().sum::() / M as f64); + for (k, w) in self.weights.iter_mut().enumerate() { + let freq = k as f64 * self.df; + let a = MicArray::steering_from_distances(distances, freq); + // Keep the common delay: w^H gets e^{-j omega tau_ref} + let keep_delay = Complex64::from_polar(1.0, 2.0 * PI * freq * delay_ref); + *w = a.unscale(a.norm_squared()) * keep_delay; + } + } + + /// Add one STFT frame (all N/2 + 1 bins); appends N/2 / factor samples to the output. + pub fn push_frame(&mut self, snapshots: &[SVector]) { + assert_eq!(snapshots.len(), self.weights.len(), "one snapshot per bin"); + + for ((y, w), x) in self.spectrum.iter_mut().zip(&self.weights).zip(snapshots) { + *y = w.dotc(x); + } + // A real signal has real DC and Nyquist bins; the inverse FFT requires it + let last = self.spectrum.len() - 1; + self.spectrum[0].im = 0.0; + self.spectrum[last].im = 0.0; + + self.plan + .process_with_scratch(&mut self.spectrum, &mut self.segment, &mut self.scratch) + .expect("buffers have the planned lengths"); + + for (acc, s) in self.ola.iter_mut().zip(&self.segment) { + *acc += s * self.gain; + } + let hop = self.ola.len() / 2; + self.decimator.push(&self.ola[..hop], &mut self.output); + self.ola.copy_within(hop.., 0); + self.ola[hop..].fill(0.0); + } + + /// Samples at [`TRACK_SAMPLE_RATE`] since the previous call + pub fn take_output(&mut self) -> Vec { + std::mem::take(&mut self.output) + } +} + +/// Low-pass FIR and every `factor`-th sample, streaming +struct Decimator { + taps: Vec, + factor: usize, + /// The last `taps.len() - 1` input samples, then new input + buf: Vec, + /// Index in `buf` of the next output sample + next: usize, +} + +/// FIR length per decimation factor; 95 taps at factor 3 give a transition of about 2.8 kHz +const TAPS_PER_FACTOR: usize = 32; +/// Cutoff relative to the output rate: 0.4 * 16 kHz = 6.4 kHz, stopband from about 7.8 kHz +const CUTOFF: f64 = 0.4; + +impl Decimator { + fn new(factor: usize) -> Self { + let taps = if factor == 1 { + vec![1.0] + } else { + lowpass_taps(TAPS_PER_FACTOR * factor - 1, CUTOFF / factor as f64) + }; + let history = taps.len() - 1; + Self { + taps, + factor, + buf: vec![0.0; history], + next: history, + } + } + + fn push(&mut self, input: &[f64], out: &mut Vec) { + self.buf.extend_from_slice(input); + while self.next < self.buf.len() { + let y: f64 = self + .taps + .iter() + .enumerate() + .map(|(j, h)| h * self.buf[self.next - j]) + .sum(); + out.push(y as f32); + self.next += self.factor; + } + // Keep the history the filter needs for the next output + let remove = self.buf.len() - (self.taps.len() - 1); + self.buf.drain(..remove); + self.next -= remove; + } +} + +/// Windowed-sinc low-pass (Blackman), DC gain 1; `cutoff` relative to the sample rate +fn lowpass_taps(len: usize, cutoff: f64) -> Vec { + let center = (len - 1) as f64 / 2.0; + let taps: Vec = (0..len) + .map(|n| { + let t = n as f64 - center; + let sinc = if t == 0.0 { + 2.0 * cutoff + } else { + (2.0 * PI * cutoff * t).sin() / (PI * t) + }; + let x = 2.0 * PI * n as f64 / (len - 1) as f64; + let blackman = 0.42 - 0.5 * x.cos() + 0.08 * (2.0 * x).cos(); + sinc * blackman + }) + .collect(); + let sum: f64 = taps.iter().sum(); + taps.iter().map(|h| h / sum).collect() +} + +#[cfg(test)] +mod test { + use std::f64::consts::PI; + + use super::{BeamSignal, Decimator, TRACK_SAMPLE_RATE, decimation_factor}; + use crate::{ + Point3, distance_to_delay, + dsp::stft::{Stft, StftStream}, + geometry::mic_array::MicArray, + source::simulator::{ + Simulator, + scene::{MicNoiseParams, Scene, Source}, + signal::{Noise, NoiseParams, Signal, Sine}, + }, + test_util::{FFT_LEN, SAMPLE_RATE, read_all}, + }; + + /// Scene of bin-centered sines (bin, position), no mic noise + fn sine_scene(sines: &[(usize, Point3)], duration_s: f64) -> Scene { + Scene { + sample_rate: SAMPLE_RATE, + duration_s, + mic_noise_params: MicNoiseParams { + seed: 1, + level_db: -1000.0, + }, + sources: sines + .iter() + .map(|&(bin, pos)| Source { + pos, + signal: Signal::Sine(Sine::new(1.0, bin_freq(bin), 0.0)), + }) + .collect(), + } + } + + fn bin_freq(bin: usize) -> f64 { + bin as f64 * SAMPLE_RATE / FFT_LEN as f64 + } + + /// Beam signal toward `target` over the whole scene, with decimation `factor` + fn beam_output(scene: &Scene, target: &Point3, factor: usize) -> Vec { + let mic_array = MicArray::uma16(); + let samples = read_all( + &mut Simulator::new(scene.clone(), mic_array.clone()).unwrap(), + 4096, + ); + let stft = Stft::new(FFT_LEN); + let sample_rate = factor as f64 * TRACK_SAMPLE_RATE; + let mut beam = BeamSignal::new( + FFT_LEN, + stft.scale(), + sample_rate, + &mic_array.get_distance_from(target), + ); + let mut stream = StftStream::<16>::new(stft); + stream.push(&samples, |snapshots, _l| beam.push_frame(snapshots)); + beam.take_output() + } + + /// Amplitude of the component at `freq` (least squares fit of sine and cosine) + fn amplitude_at(signal: &[f32], sample_rate: f64, freq: f64) -> f64 { + let (mut c, mut s) = (0.0, 0.0); + for (n, &y) in signal.iter().enumerate() { + let phase = 2.0 * PI * freq * n as f64 / sample_rate; + c += y as f64 * phase.cos(); + s += y as f64 * phase.sin(); + } + 2.0 * (c * c + s * s).sqrt() / signal.len() as f64 + } + + #[test] + fn decimation_factors() { + assert_eq!(decimation_factor(48_000.0), Some(3)); + assert_eq!(decimation_factor(16_000.0), Some(1)); + assert_eq!(decimation_factor(44_100.0), None); + assert_eq!(decimation_factor(8_000.0), None); + } + + /// Without decimation, a source at the beam point comes out as s(t - r_ref / c) (referenced + /// to 1 m, r_ref the mean mic distance) once the first frame has passed. Band noise at + /// off-bin frequencies is not periodic in the frame, so a circular shift that wraps the + /// window would show up here (a bin-centered sine hides it). + #[test] + fn reconstructs_source_signal() { + let pos = Point3::new(0.4, 0.2, 2.0); + let signal = Signal::Noise(Noise::from(NoiseParams { + seed: 3, + n_components: 200, + f_lo_hz: 200.0, + f_hi_hz: 6000.0, + level_db: 94.0, + })); + // Factor 1 needs a 16 kHz scene + let mut scene = sine_scene(&[(50, pos)], 1.0); + scene.sample_rate = TRACK_SAMPLE_RATE; + scene.sources[0].signal = signal.clone(); + + let output = beam_output(&scene, &pos, 1); + let distances = MicArray::uma16().get_distance_from(&pos); + let delay_ref = distance_to_delay(distances.iter().sum::() / 16.0); + let (mut error, mut power) = (0.0, 0.0); + for (n, &y) in output.iter().enumerate().skip(FFT_LEN) { + let expected = signal.eval(n as f64 / TRACK_SAMPLE_RATE - delay_ref); + error += (y as f64 - expected).powi(2); + power += expected.powi(2); + } + let relative = (error / power).sqrt(); + // About 1e-7 (wrap at the window tails); 2e-2 with the whole delay removed + assert!(relative < 1e-5, "relative error {relative}"); + } + + /// At 48 kHz the decimated output keeps the source amplitude. + #[test] + fn decimated_amplitude() { + let pos = Point3::new(0.1, 0.2, 1.0); + let scene = sine_scene(&[(50, pos)], 0.5); + let output = beam_output(&scene, &pos, 3); + // N/2 samples at 48 kHz per frame, a third of them at 16 kHz + let skip = FFT_LEN / 2; + let amplitude = amplitude_at(&output[skip..], TRACK_SAMPLE_RATE, bin_freq(50)); + assert!((amplitude - 1.0).abs() < 1e-3, "amplitude {amplitude}"); + } + + /// Two sources: the beam toward one attenuates the other. + #[test] + fn attenuates_other_direction() { + let (pos_a, pos_b) = (Point3::new(-0.6, 0.0, 1.0), Point3::new(0.6, 0.0, 1.0)); + let (bin_a, bin_b) = (70, 75); + let scene = sine_scene(&[(bin_a, pos_a), (bin_b, pos_b)], 0.5); + + let output = beam_output(&scene, &pos_a, 3); + let skip = FFT_LEN / 2; + let a = amplitude_at(&output[skip..], TRACK_SAMPLE_RATE, bin_freq(bin_a)); + let b = amplitude_at(&output[skip..], TRACK_SAMPLE_RATE, bin_freq(bin_b)); + // Both sources have amplitude 1 at 1 m; B is about 1.17 m from the array center + let suppression_db = 20.0 * (a / b).log10(); + // About 17 dB at 3.4 kHz with sources 62 degrees apart + assert!( + suppression_db > 15.0, + "A {a}, B {b}: {suppression_db:.1} dB" + ); + } + + /// The decimation filter passes speech frequencies and blocks what would alias. + #[test] + fn decimator_passband_and_stopband() { + let gain = |freq: f64| { + let input: Vec = (0..48_000) + .map(|n| (2.0 * PI * freq * n as f64 / SAMPLE_RATE).sin()) + .collect(); + let mut decimator = Decimator::new(3); + let mut out = Vec::new(); + // Uneven chunks: the stream must not depend on the chunking + for chunk in input.chunks(500) { + decimator.push(chunk, &mut out); + } + assert_eq!(out.len(), 16_000); + amplitude_at(&out[200..], TRACK_SAMPLE_RATE, freq) + }; + + assert!((gain(1000.0) - 1.0).abs() < 1e-3, "1 kHz: {}", gain(1000.0)); + assert!((gain(5000.0) - 1.0).abs() < 1e-2, "5 kHz: {}", gain(5000.0)); + // 12 kHz would alias to 4 kHz; amplitude_at(.., 12 kHz) at 16 kHz measures that alias + let stop = gain(12_000.0); + assert!(stop < 1e-4, "12 kHz: {stop}"); + } +} diff --git a/crates/core/src/dsp/beamformer.rs b/crates/core/src/dsp/beamformer.rs index 1b134e8..9ae8270 100644 --- a/crates/core/src/dsp/beamformer.rs +++ b/crates/core/src/dsp/beamformer.rs @@ -13,6 +13,7 @@ pub mod functional; /// Steering vectors for all scan grid points; the geometry is precomputed, the frequency part is /// built per call +#[derive(Clone)] pub struct Steering { /// Precomputed distances per grid point distances: Vec<[f64; M]>, @@ -66,6 +67,56 @@ impl Steering { } } +/// Power arriving coherently from grid point `point`, summed over `bins`, and the mean mic +/// auto-power sum_m R_mm / M over `bins` (all the received power). +/// +/// The coherent power is the delay-and-sum power with diagonal removal, P = c (a^H R a - +/// sum_m g_m R_mm), with c such that a lone source at the point gives its mean power at the mics +/// (R = S a a^H gives S sum(g)/M). A source elsewhere gives its power times the sidelobe level, +/// noise that is uncorrelated between the mics about 0 (also negative), whatever its level. +/// Independent of the beamformer that made the map, so a peak of any map can be checked +/// against it. +pub fn coherent_and_received_power( + cov: &[SMatrix], + bins: RangeInclusive, + steering: &Steering, + point: usize, +) -> (f64, f64) { + debug_assert!(M > 1, "diagonal removal needs at least two mics"); + let gains = steering.gains(point); + let sum_g: f64 = gains.iter().sum(); + let sum_g2: f64 = gains.iter().map(|g| g * g).sum(); + let c = sum_g / M as f64 / (sum_g * sum_g - sum_g2); + + let (mut coherent, mut received) = (0.0, 0.0); + for bin in bins { + let r = &cov[bin]; + let a = steering.vector(point, bin); + let quad = a.dotc(&(r * a)).re; + let diag: f64 = (0..M).map(|m| gains[m] * r[(m, m)].re).sum(); + coherent += c * (quad - diag); + received += (0..M).map(|m| r[(m, m)].re).sum::() / M as f64; + } + (coherent, received) +} + +/// Fraction of the power received over `bins` that arrives coherently from `point`, see +/// [`coherent_and_received_power`]: about 1 for a lone source at the point, the sidelobe level +/// for a source elsewhere, about 0 for uncorrelated noise. 0 without received power. +pub fn source_fraction( + cov: &[SMatrix], + bins: RangeInclusive, + steering: &Steering, + point: usize, +) -> f64 { + let (coherent, received) = coherent_and_received_power(cov, bins, steering, point); + if received > 0.0 { + coherent / received + } else { + 0.0 + } +} + pub trait Beamformer { /// Get the number of points in the scanning grid fn n_points(&self) -> usize; @@ -92,3 +143,49 @@ pub enum Normalization { /// c = 1/(M a^H a): peak exactly at the source, P = S * (a^H a)/M there TrueLocation, } + +#[cfg(test)] +mod test { + use nalgebra::SMatrix; + + use super::{Steering, coherent_and_received_power, source_fraction}; + use crate::{ + Complex64, + geometry::mic_array::MicArray, + test_util::{BIN, FFT_LEN, SAMPLE_RATE, test_scan_grid}, + }; + + /// A source at the point with mic noise at the same level gives 1/2 exactly (the diagonal + /// removal cancels the noise, the mic power counts it); noise alone gives 0; a source + /// elsewhere gives less than at the point. + #[test] + fn fraction_of_source_and_noise() { + let grid = test_scan_grid(); + let steering = Steering::new(&grid, &MicArray::uma16(), SAMPLE_RATE, FFT_LEN); + let point = grid.index(grid.height() / 2, grid.width() / 2); + let a = steering.vector(point, BIN); + let gains = steering.gains(point); + let mean_mic_power = gains.iter().sum::() / 16.0; + let identity = SMatrix::::identity(); + + let mut cov = vec![SMatrix::::zeros(); FFT_LEN / 2 + 1]; + cov[BIN] = a * a.adjoint() + identity * Complex64::from(mean_mic_power); + let (coherent, received) = coherent_and_received_power(&cov, BIN..=BIN, &steering, point); + assert!( + (coherent - mean_mic_power).abs() < 1e-9, + "coherent {coherent}" + ); + assert!( + (received - 2.0 * mean_mic_power).abs() < 1e-9, + "received {received}" + ); + let at_source = source_fraction(&cov, BIN..=BIN, &steering, point); + assert!((at_source - 0.5).abs() < 1e-9, "at the source: {at_source}"); + let corner = source_fraction(&cov, BIN..=BIN, &steering, 0); + assert!(corner < 0.5 * at_source, "corner: {corner}"); + + cov[BIN] = identity * Complex64::from(mean_mic_power); + let noise = source_fraction(&cov, BIN..=BIN, &steering, point); + assert!(noise.abs() < 1e-12, "noise only: {noise}"); + } +} diff --git a/crates/core/src/dsp/beamformer/clean_sc.rs b/crates/core/src/dsp/beamformer/clean_sc.rs index fef6bd0..5d78b48 100644 --- a/crates/core/src/dsp/beamformer/clean_sc.rs +++ b/crates/core/src/dsp/beamformer/clean_sc.rs @@ -26,7 +26,11 @@ use crate::{ /// it would add its noise or reflection maximum. /// /// The clean maps of all bins are summed (bins in parallel) and convolved with a Gaussian clean -/// beam. Without diagonal removal, mic noise sigma^2 I is a floor in every dirty map. +/// beam. With `equal_bins` each bin's clean map is divided by its maximum first (bins without a +/// source stay zero), so every bin with a source has the same say: otherwise the loudest bins +/// decide, and those are the low ones, where the array cannot separate a source from its +/// reflection off a wall or ceiling and the maximum lands anywhere between. Without diagonal +/// removal, mic noise sigma^2 I is a floor in every dirty map. pub struct CleanSc { steering: Steering, /// Loop gain in (0, 1] @@ -39,6 +43,8 @@ pub struct CleanSc { beam_sigma: f64, /// Zero the diagonal of R (removes mic self-noise) remove_diag: bool, + /// Divide each bin's clean map by its maximum before summing + equal_bins: bool, /// 1 / (|a| sqrt(M)), or sqrt(M - 1) with diagonal removal, per grid point; scales a to w weight_scale: Vec, } @@ -61,6 +67,7 @@ impl CleanSc { /// - `stop_db`: stop when the maximum is this many dB below the largest one of all bins, >= 0 /// - `beam_sigma`: clean beam standard deviation in grid cells, >= 0 /// - `remove_diag`: zero the diagonal of R, removes mic self-noise + /// - `equal_bins`: divide each bin's clean map by its maximum before summing /// /// Panics on values outside these ranges. pub fn new( @@ -70,6 +77,7 @@ impl CleanSc { stop_db: f64, beam_sigma: f64, remove_diag: bool, + equal_bins: bool, ) -> Self { assert!(gain > 0.0 && gain <= 1.0, "gain must be in (0, 1]"); assert!(max_iter > 0, "max_iter must be at least 1"); @@ -91,6 +99,7 @@ impl CleanSc { stop_ratio: 10f64.powf(-stop_db / 10.0), beam_sigma, remove_diag, + equal_bins, weight_scale, } } @@ -227,7 +236,19 @@ impl Beamformer for CleanSc { .fold( || vec![0.0; n], |mut clean, bin| { - self.clean_bin(bin, threshold, &mut clean); + if self.equal_bins { + let mut single = vec![0.0; n]; + self.clean_bin(bin, threshold, &mut single); + let max = single.iter().copied().fold(0.0, f64::max); + if max > 0.0 { + clean + .iter_mut() + .zip(&single) + .for_each(|(c, s)| *c += s / max); + } + } else { + self.clean_bin(bin, threshold, &mut clean); + } clean }, ) @@ -307,17 +328,29 @@ mod test { MicArray::uma16().steering(&scan_grid.points()[i], f) } - /// Clean map of `cov` at bins `bins`, no clean beam + /// Clean map of `cov` at bins `bins`, no clean beam, bins summed as they are fn clean_map( cov: &[SMatrix], bins: std::ops::RangeInclusive, gain: f64, stop_db: f64, remove_diag: bool, + ) -> Vec { + clean_map_equal(cov, bins, gain, stop_db, remove_diag, false) + } + + /// As `clean_map`, with or without `equal_bins` + fn clean_map_equal( + cov: &[SMatrix], + bins: std::ops::RangeInclusive, + gain: f64, + stop_db: f64, + remove_diag: bool, + equal_bins: bool, ) -> Vec { let scan_grid = test_scan_grid(); let steering = Steering::new(&scan_grid, &MicArray::uma16(), SAMPLE_RATE, FFT_LEN); - let clean_sc = CleanSc::new(steering, gain, 30, stop_db, 0.0, remove_diag); + let clean_sc = CleanSc::new(steering, gain, 30, stop_db, 0.0, remove_diag, equal_bins); let mut power = vec![0.0; scan_grid.points().len()]; clean_sc.power_map(cov, bins, &mut power); power @@ -418,6 +451,38 @@ mod test { assert!(power[i_1] > 0.0, "weak bin not cleaned at 40 dB"); } + /// With equal bins the 30 dB weaker source gets the same height as the strong one (each + /// bin's map is divided by its maximum: 1 at the source with gain 1 and no clean beam); a + /// bin that cleaned nothing (below the threshold) still adds nothing. + #[test] + fn equal_bins_level_the_bins() { + let scan_grid = test_scan_grid(); + let (i_0, i_1) = (scan_grid.index(40, 100), scan_grid.index(20, 30)); + let weak_bin = BIN + 10; + let a_0 = steering_at(&scan_grid, i_0, BIN); + let a_1 = steering_at(&scan_grid, i_1, weak_bin); + let mut cov = vec![SMatrix::zeros(); FFT_LEN / 2 + 1]; + cov[BIN] = a_0 * a_0.adjoint(); + cov[weak_bin] = (a_1 * a_1.adjoint()).scale(1e-3); + + let power = clean_map_equal(&cov, BIN..=weak_bin, 1.0, 40.0, true, true); + assert!( + (power[i_0] - 1.0).abs() < 1e-9, + "strong source {}", + power[i_0] + ); + assert!( + (power[i_1] - 1.0).abs() < 1e-9, + "weak source {}", + power[i_1] + ); + let rest: f64 = power.iter().sum::() - 2.0; + assert!(rest.abs() < 1e-9, "elsewhere {rest}"); + + let power = clean_map_equal(&cov, BIN..=weak_bin, 1.0, 20.0, true, true); + assert_eq!(power[i_1], 0.0, "weak bin below the threshold"); + } + /// Two incoherent sources far apart: the strongest clean point of each image half is at /// (or next to) its source. #[test] @@ -425,7 +490,7 @@ mod test { let scan_grid = test_scan_grid(); let mic_array = MicArray::uma16(); let steering = Steering::new(&scan_grid, &mic_array, SAMPLE_RATE, FFT_LEN); - let clean_sc = CleanSc::new(steering, 0.9, 30, 30.0, 0.0, true); + let clean_sc = CleanSc::new(steering, 0.9, 30, 30.0, 0.0, true, false); let row = scan_grid.height() / 2; let (col_1, col_2) = (scan_grid.width() / 6, 5 * scan_grid.width() / 6); diff --git a/crates/core/src/dsp/beamformer/delay_and_sum.rs b/crates/core/src/dsp/beamformer/delay_and_sum.rs index d6d2973..c949a0b 100644 --- a/crates/core/src/dsp/beamformer/delay_and_sum.rs +++ b/crates/core/src/dsp/beamformer/delay_and_sum.rs @@ -120,6 +120,7 @@ mod test { use nalgebra::SMatrix; use crate::{ + Complex64, dsp::{ beamformer::{ Beamformer as _, Steering, @@ -171,7 +172,7 @@ mod test { for (power, r) in power.iter().zip(scan_grid.points()) { let a_i_norm_squared: f64 = mic_array - .get_distance_from(&r) + .get_distance_from(r) .iter() .map(|r| r.powi(-2)) .sum(); @@ -274,33 +275,129 @@ mod test { /// Expect P(p_0) = 1: checks the new normalization and the diagonal term together. #[test] fn diag_removal_hand_built_r_at_source() { - // 1. Same setup as `hand_built_r_peak_at_source`, but `Das::new(..., true)` - // 2. power_map over BIN..=BIN - // 3. assert P(p_0) = 1 - todo!() + let (row, col) = (10, 20); + let scan_grid = test_scan_grid(); + let mic_array = MicArray::uma16(); + let steering = Steering::new(&scan_grid, &mic_array, SAMPLE_RATE, FFT_LEN); + let das_beamformer = Das::new(steering, true, Normalization::TrueLevel); + + let p_0 = scan_grid.points()[scan_grid.index(row, col)]; + let a_0 = mic_array.steering(&p_0, F_SINE); + let mut cov = vec![SMatrix::zeros(); FFT_LEN / 2 + 1]; + cov[BIN] = a_0 * a_0.adjoint(); + + let mut power = vec![0.0; scan_grid.points().len()]; + das_beamformer.power_map(&cov, BIN..=BIN, &mut power); + + let p = power[scan_grid.index(row, col)]; + assert!((p - 1.0).abs() < 1e-12, "P(p_0) should be 1, got {p}"); } /// R = sigma^2 I in one bin, nothing else. /// Expect P = 0 everywhere with diagonal removal, sigma^2 / (a^H a) without. #[test] fn diag_removal_noise_only_is_zero() { - // 1. cov[BIN] = sigma^2 * I, zeros elsewhere - // 2. power_map with `true`: assert |P| < tol everywhere (tol relative to sigma^2) - // 3. power_map with `false`: assert P = sigma^2 / sum(1/r_m^2) per point - todo!() + let sigma_sq = 3.0; + let scan_grid = test_scan_grid(); + let mic_array = MicArray::uma16(); + let steering = Steering::new(&scan_grid, &mic_array, SAMPLE_RATE, FFT_LEN); + let mut cov = vec![SMatrix::zeros(); FFT_LEN / 2 + 1]; + cov[BIN] = SMatrix::identity() * Complex64::new(sigma_sq, 0.0); + + let mut power = vec![0.0; scan_grid.points().len()]; + Das::new(steering.clone(), true, Normalization::TrueLevel).power_map( + &cov, + BIN..=BIN, + &mut power, + ); + for (p, r) in power.iter().zip(scan_grid.points()) { + assert!(p.abs() < 1e-12 * sigma_sq, "P at {r} should be 0, got {p}"); + } + + Das::new(steering, false, Normalization::TrueLevel).power_map(&cov, BIN..=BIN, &mut power); + for (p, r) in power.iter().zip(scan_grid.points()) { + let sum_g: f64 = mic_array + .get_distance_from(r) + .iter() + .map(|r_m| r_m.powi(-2)) + .sum(); + let expected = sigma_sq / sum_g; + assert!( + (p - expected).abs() < 1e-12 * expected, + "P at {r}: expected {expected}, got {p}" + ); + } } /// Oracle R vs stream R_est with strong mic noise, diagonal removal on. /// Expect the peak at the source for both, and P_est close to P_ref = S. #[test] fn diag_removal_stream_r_peak_at_source() { - // 1. Source on the center grid point, 110 dB mic noise - // 2. cov_ref = reference_covariance(...), (cov_est, l) = stream_covariance(...) - // 3. power_map for both with `true` - // 4. assert P_ref(source) = sine_bin_power (sigma^2 I is gone) - // 5. assert argmax of P_est == source index - // 6. assert P_est(source) close to P_ref(source) - // 7. optional: compare with `false`, |P_est - P_ref| should be smaller with removal - todo!() + const DURATION_S: f64 = 1.0; + let scan_grid = test_scan_grid(); + let mut scene = test_scene(110.0, DURATION_S); + let idx_source = scan_grid.index(scan_grid.height() / 2, scan_grid.width() / 2); + scene.sources[0].pos = scan_grid.points()[idx_source]; + let mic_array = MicArray::uma16(); + let steering = Steering::new(&scan_grid, &mic_array, SAMPLE_RATE, FFT_LEN); + + let cov_ref = reference_covariance(&scene, &mic_array, &Stft::new(FFT_LEN)); + let (cov_est, l) = stream_covariance(&scene, &mic_array); + assert!(l > 0, "no STFT frames"); + + let argmax = |power: &[f64]| { + power + .iter() + .enumerate() + .max_by(|a, b| a.1.total_cmp(b.1)) + .unwrap() + .0 + }; + let mut power_ref = vec![0.0; scan_grid.points().len()]; + let mut power_est = vec![0.0; scan_grid.points().len()]; + let with_removal = Das::new(steering.clone(), true, Normalization::TrueLevel); + with_removal.power_map(&cov_ref, BIN..=BIN, &mut power_ref); + with_removal.power_map(&cov_est, BIN..=BIN, &mut power_est); + + // sigma^2 I is gone from the oracle map: exactly the sine power at the source + let expected = sine_bin_power(scene.sources[0].signal.components()[0].amplitude, FFT_LEN); + let p_ref = power_ref[idx_source]; + assert!( + (p_ref - expected).abs() < 1e-9 * expected, + "P_ref at source: expected {expected}, got {p_ref}" + ); + assert_eq!( + argmax(&power_ref), + idx_source, + "oracle maximum off the source" + ); + // The main lobe spans dozens of cells of this fine grid and its top is flat to well + // below a percent between neighbors, so the estimation noise moves the maximum by a + // cell + let idx_est = argmax(&power_est); + let width = scan_grid.width(); + let (col_est, row_est) = (idx_est % width, idx_est / width); + let (col, row) = (idx_source % width, idx_source / width); + assert!( + col_est.abs_diff(col) <= 1 && row_est.abs_diff(row) <= 1, + "stream maximum at ({col_est}, {row_est}), source at ({col}, {row})" + ); + // Only the estimation noise remains, about sqrt(2 sigma^2 / (|a|^2 S L)) relative + let p_est = power_est[idx_source]; + assert!( + (p_est - p_ref).abs() < 0.05 * p_ref, + "P_est at source: expected {p_ref}, got {p_est}" + ); + + // Without removal the noise adds sigma^2 / sum(g): a larger error than with removal + let without = Das::new(steering, false, Normalization::TrueLevel); + let mut power_plain = vec![0.0; scan_grid.points().len()]; + without.power_map(&cov_est, BIN..=BIN, &mut power_plain); + let plain_error = (power_plain[idx_source] - expected).abs(); + assert!( + (p_est - expected).abs() < plain_error, + "removal error {} not below {plain_error}", + (p_est - expected).abs() + ); } } diff --git a/crates/core/src/dsp/covariance.rs b/crates/core/src/dsp/covariance.rs index 9193685..83b8fd3 100644 --- a/crates/core/src/dsp/covariance.rs +++ b/crates/core/src/dsp/covariance.rs @@ -223,9 +223,9 @@ mod tests { cov.accumulate(&y); } - for k in 0..N_BINS { - let target = y[k] * y[k].adjoint(); - let error = (cov.estimate()[k] - target).norm() / target.norm(); + for (k, (estimate, y_k)) in cov.estimate().iter().zip(&y).enumerate() { + let target = y_k * y_k.adjoint(); + let error = (estimate - target).norm() / target.norm(); assert!(error < 1e-8, "bin {k}: relative error {error}"); } } diff --git a/crates/core/src/dsp/stft.rs b/crates/core/src/dsp/stft.rs index 84b5f6b..28ea783 100644 --- a/crates/core/src/dsp/stft.rs +++ b/crates/core/src/dsp/stft.rs @@ -13,8 +13,8 @@ pub struct Stft { input_buf: Vec, output_buf: Vec, window: Vec, - /// Kept for debugging and testing - _scale: f64, + /// 1/sqrt(sum(window^2)), part of `weights` + scale: f64, weights: Vec, } @@ -38,7 +38,7 @@ impl Stft { output_buf: plan.make_output_vec(), plan, window: hann_window, - _scale: scale, + scale, weights, } } @@ -51,6 +51,11 @@ impl Stft { &self.weights } + /// Scale factor 1/sqrt(sum(window^2)) applied with the window + pub fn scale(&self) -> f64 { + self.scale + } + pub fn transform(&mut self, frame: &[f64]) -> &[Complex64] { assert_eq!( frame.len(), @@ -105,8 +110,8 @@ impl StftStream { let fft_len = self.stft.fft_len(); for frame in chunk.as_chunks::().0 { - for m in 0..M { - self.buf[m][self.fill] = frame[m]; + for (buf, &x_m) in self.buf.iter_mut().zip(frame) { + buf[self.fill] = x_m; } self.fill += 1; @@ -205,7 +210,7 @@ mod test { // Scaling factor should be approx sqrt(8 / 3N) assert!( - (stft._scale - (8.0 / 3.0 / fft_len as f64).sqrt()).abs() < 1e-12, + (stft.scale() - (8.0 / 3.0 / fft_len as f64).sqrt()).abs() < 1e-12, "Scaling factor should be sqrt(8 / 3N)" ) } diff --git a/crates/core/src/geometry/mic_array.rs b/crates/core/src/geometry/mic_array.rs index b7e30a3..1c024c8 100644 --- a/crates/core/src/geometry/mic_array.rs +++ b/crates/core/src/geometry/mic_array.rs @@ -174,7 +174,7 @@ mod test { let tau_mk_approx = p .map(|p_m| p_m.coords.dot(&u_k)) .map(distance_to_delay) - .map(|x| -1.0 * x + distance_to_delay(DISTANCE)); + .map(|x| -x + distance_to_delay(DISTANCE)); println!("tau_mk: {tau_mk:#?}, tau_mk_approx: {tau_mk_approx:#?}"); diff --git a/crates/core/src/geometry/scan_grid.rs b/crates/core/src/geometry/scan_grid.rs index 03a0bbf..6f202c8 100644 --- a/crates/core/src/geometry/scan_grid.rs +++ b/crates/core/src/geometry/scan_grid.rs @@ -1,12 +1,58 @@ use crate::{Point3, geometry::camera::Camera}; +/// Scan grid cells farther off the camera axis than this are never added as margin: the pinhole +/// plane grows as tan(angle), and the array has no useful resolution at grazing angles +pub const MAX_OFF_AXIS_DEG: f64 = 55.0; + +/// A rectangle of scan grid cells: the part of the grid that the camera sees +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct Viewport { + /// First visible column and row in the full grid + pub col: usize, + pub row: usize, + /// Visible size in cells + pub width: usize, + pub height: usize, +} + +impl Viewport { + /// The whole grid + pub fn full(width: usize, height: usize) -> Self { + Self { + col: 0, + row: 0, + width, + height, + } + } + + /// Whether (col, row) in full grid cells (sub-cell positions allowed) lies within the visible + /// cell centers + pub fn contains(&self, col: f64, row: f64) -> bool { + let (c, r) = (col - self.col as f64, row - self.row as f64); + 0.0 <= c && c <= (self.width - 1) as f64 && 0.0 <= r && r <= (self.height - 1) as f64 + } + + /// Copy the visible part of `full` (row-major, `full_width` wide) into `out` + /// (`width * height`) + pub fn crop(&self, full: &[f64], full_width: usize, out: &mut [f64]) { + assert_eq!(out.len(), self.width * self.height); + for (row, out) in out.chunks_exact_mut(self.width).enumerate() { + let start = (self.row + row) * full_width + self.col; + out.copy_from_slice(&full[start..start + self.width]); + } + } +} + pub struct ScanGrid { - /// Scan grid width in "pixels" + /// Scan grid width in "pixels", including the margin width: usize, - /// Scan grid height in "pixels" + /// Scan grid height in "pixels", including the margin height: usize, /// Points of the grid stored in row-major form points: Vec, + /// The cells inside the camera image + viewport: Viewport, } impl ScanGrid { @@ -15,22 +61,44 @@ impl ScanGrid { /// - `width` is the grid's resolution along the x-axis /// - `z` is the depth from the camera in meters pub fn new(camera: &Camera, width: u32, z: f64) -> Self { + Self::with_margin(camera, width, z, 0.0) + } + + /// Like [`new`](Self::new), extended on every side by `margin` radians beyond the field of + /// view (as far as [`MAX_OFF_AXIS_DEG`] allows), at the same cell size. + /// + /// A source outside the image then has its maximum at its own direction instead of on the + /// image border, and a deconvolving beamformer (CLEAN-SC) takes its contribution out of + /// the visible part. The visible cells are [`viewport`](Self::viewport). + pub fn with_margin(camera: &Camera, width: u32, z: f64, margin: f64) -> Self { let scaled_camera = Camera { intrinsics: camera.intrinsics.scaled(width), mount: camera.mount.clone(), }; - let height = scaled_camera.intrinsics.height; + let intrinsics = &scaled_camera.intrinsics; + let height = intrinsics.height; + let margin_cols = margin_cells(intrinsics.fx, intrinsics.cx, width, margin); + let margin_rows = margin_cells(intrinsics.fy, intrinsics.cy, height, margin); + let cols = -(margin_cols as i64)..(width as i64 + margin_cols as i64); + let rows = -(margin_rows as i64)..(height as i64 + margin_rows as i64); // workaround so `move` copies the reference let scaled_camera_ref = &scaled_camera; - let points = Vec::from_iter((0..height).flat_map(|row| { - (0..width).map(move |col| scaled_camera_ref.unproject(col as f64, row as f64, z)) + let points = Vec::from_iter(rows.clone().flat_map(|row| { + cols.clone() + .map(move |col| scaled_camera_ref.unproject(col as f64, row as f64, z)) })); Self { - width: width as usize, - height: height as usize, + width: width as usize + 2 * margin_cols, + height: height as usize + 2 * margin_rows, points, + viewport: Viewport { + col: margin_cols, + row: margin_rows, + width: width as usize, + height: height as usize, + }, } } @@ -46,15 +114,32 @@ impl ScanGrid { &self.points } + /// The cells inside the camera image (all of them without margin) + pub fn viewport(&self) -> Viewport { + self.viewport + } + pub fn index(&self, row: usize, col: usize) -> usize { debug_assert!(row < self.height && col < self.width); row * self.width + col } } +/// Cells to add beyond the outermost cell center of an axis with focal length `f` and principal +/// point `c` (grid pixels, `n` cells) so that the grid reaches `margin` radians further, but +/// not beyond `MAX_OFF_AXIS_DEG` +fn margin_cells(f: f64, c: f64, n: u32, margin: f64) -> usize { + let edge = ((n as f64 - 1.0 - c) / f).atan(); + let outer = (edge + margin).min(MAX_OFF_AXIS_DEG.to_radians()).max(edge); + (f * (outer.tan() - edge.tan())).round() as usize +} + #[cfg(test)] mod test { - use crate::{geometry::scan_grid::ScanGrid, test_util::test_camera}; + use crate::{ + geometry::scan_grid::{MAX_OFF_AXIS_DEG, ScanGrid, Viewport}, + test_util::test_camera, + }; #[test] fn points_len_w_times_h() { @@ -123,4 +208,70 @@ mod test { } } } + + /// With a margin the visible cells are the plain grid, shifted by the margin; the margin + /// cells continue the pixel raster beyond the image. + #[test] + fn margin_keeps_visible_cells() { + let camera = test_camera(); + let plain = ScanGrid::new(&camera, 64, 2.0); + let grid = ScanGrid::with_margin(&camera, 64, 2.0, 15f64.to_radians()); + let vp = grid.viewport(); + + assert_eq!((vp.width, vp.height), (plain.width(), plain.height())); + assert!(vp.row > 0, "a 15 degree margin adds rows: {vp:?}"); + assert_eq!(grid.width(), vp.width + 2 * vp.col); + assert_eq!(grid.height(), vp.height + 2 * vp.row); + assert_eq!(grid.points().len(), grid.width() * grid.height()); + for row in 0..vp.height { + for col in 0..vp.width { + let a = plain.points()[plain.index(row, col)]; + let b = grid.points()[grid.index(row + vp.row, col + vp.col)]; + assert!((a - b).norm() < 1e-9, "({row}, {col}): {a} vs {b}"); + } + } + // The cell above the image projects one grid pixel above it + let (_, v) = camera.project(&grid.points()[grid.index(vp.row - 1, vp.col)]); + let s_y = plain.height() as f64 / camera.intrinsics.height as f64; + assert!((v - (-0.5 / s_y - 0.5)).abs() < 1e-9, "v = {v}"); + } + + /// The margin ends at `MAX_OFF_AXIS_DEG`: the 110 degree test camera already sees almost + /// that far horizontally (its outermost cell center is at 54.6 degrees), so at most one + /// column is added, and a huge margin adds the same rows as the cap. + #[test] + fn margin_capped() { + let camera = test_camera(); + let grid = ScanGrid::with_margin(&camera, 64, 2.0, 15f64.to_radians()); + assert!(grid.viewport().col <= 1, "{:?}", grid.viewport()); + + let capped = ScanGrid::with_margin(&camera, 64, 2.0, 80f64.to_radians()); + let vp = capped.viewport(); + let top = capped.points()[capped.index(0, vp.col + vp.width / 2)]; + let angle = (top.y / top.z).abs().atan().to_degrees(); + assert!( + (angle - MAX_OFF_AXIS_DEG).abs() < 1.5, + "top row at {angle} degrees" + ); + assert_eq!( + ScanGrid::new(&camera, 64, 2.0).viewport(), + Viewport::full(64, 36) + ); + } + + #[test] + fn viewport_crop_and_contains() { + let vp = Viewport { + col: 1, + row: 2, + width: 2, + height: 2, + }; + let full: Vec = (0..4 * 5).map(|i| i as f64).collect(); + let mut out = vec![0.0; 4]; + vp.crop(&full, 4, &mut out); + assert_eq!(out, [9.0, 10.0, 13.0, 14.0]); + assert!(vp.contains(1.0, 2.0) && vp.contains(2.0, 3.0)); + assert!(!vp.contains(0.9, 2.0) && !vp.contains(2.0, 3.1)); + } } diff --git a/crates/core/src/lib.rs b/crates/core/src/lib.rs index f302a8a..14ffbac 100644 --- a/crates/core/src/lib.rs +++ b/crates/core/src/lib.rs @@ -4,6 +4,7 @@ pub mod peaks; pub mod pipeline; pub mod render; pub mod source; +pub mod tracking; #[cfg(test)] pub(crate) mod test_util; diff --git a/crates/core/src/peaks.rs b/crates/core/src/peaks.rs index 792a2ae..cac131c 100644 --- a/crates/core/src/peaks.rs +++ b/crates/core/src/peaks.rs @@ -2,27 +2,59 @@ use serde::Deserialize; +use crate::geometry::scan_grid::Viewport; + /// Peak search parameters, e.g. from a config file #[derive(Clone, Debug, PartialEq, Deserialize)] #[serde(default, deny_unknown_fields)] pub struct PeakConfig { /// Show peaks at all pub enabled: bool, - /// Only local maxima at most this many dB below the map maximum + /// Only local maxima at most this many dB below the map maximum (a talker next to snapping + /// is about 10 dB below it; whether a peak is a source at all is decided against the noise + /// floor, see `min_above_floor_db`) pub range_db: f64, /// Minimum distance between two peaks in grid cells; the weaker one is dropped pub min_dist_cells: f64, /// At most this many peaks, strongest first pub max_count: usize, + /// A track is shown (and recorded) only if the power arriving coherently from its direction + /// (see [`crate::dsp::beamformer::coherent_and_received_power`]) is at least this many dB + /// above the noise floor: the most coherent power from any direction while nothing is + /// there, learned from the quietest map of the last minute (see + /// [`crate::pipeline::FLOOR_WINDOW_S`]). A steady background source (fan) is the floor and + /// gets no dot. The pipeline measures it (`Track::source_db`); the app applies it when + /// showing tracks, so that it can change without a restart. + pub min_above_floor_db: f64, + /// Peaks closer than this to the border of the whole scan grid, in grid cells, are dropped + /// (0: none). Without a scan grid margin (`pipeline.grid_margin_deg` 0) the grid ends at the + /// image border, where a source outside the image has its maximum, so 1 drops those; with + /// the margin the grid border is far outside the image and a maximum there is better kept, + /// since it suppresses the leakage toward the image for `min_dist_cells` + pub edge_margin_cells: f64, + /// Extra dB a track needs above the noise floor at the left and right image border, growing + /// with the squared distance from the center (see [`center_penalty_db`]). The array resolves + /// best on its axis; toward the edges the beams widen and sidelobes of sources outside the + /// image get in (less so with the scan grid margin). Also ranks central peaks before + /// peripheral ones for `max_count`. + pub center_weight_db: f64, + /// A track is shown only if its coherent share of the power above 2 kHz + /// (`Track::source_upper_db`) is at least this, in dB: a point source or a snap reaches + /// about 0 dB, diffuse room noise and mic noise stay far below, whatever their level + pub min_coherence_db: f64, } impl Default for PeakConfig { fn default() -> Self { Self { enabled: true, - range_db: 6.0, + range_db: 12.0, min_dist_cells: 4.0, max_count: 5, + min_above_floor_db: 8.0, + edge_margin_cells: 0.0, + center_weight_db: 2.0, + min_coherence_db: -8.0, } } } @@ -45,18 +77,44 @@ pub struct Peak { /// Positions are refined with a parabola through the log power of the cell and its two /// neighbors, separately per axis (exact for a Gaussian peak). Empty if the maximum is <= 0. pub fn find_peaks(power: &[f64], [width, height]: [usize; 2], config: &PeakConfig) -> Vec { + find_peaks_in( + power, + [width, height], + Viewport::full(width, height), + config, + ) +} + +/// Like [`find_peaks`], but only the peaks inside `viewport` (the part of the scan grid the +/// camera sees), with positions and `center_weight_db` relative to it and `range_db` relative +/// to its maximum. Maxima outside still take part in the `min_dist_cells` check, so a source +/// just outside the image suppresses the local maxima its main lobe leaves near the border, +/// but they do not count toward `max_count`. `edge_margin_cells` applies to the border of the +/// whole map. +pub fn find_peaks_in( + power: &[f64], + [width, height]: [usize; 2], + viewport: Viewport, + config: &PeakConfig, +) -> Vec { assert_eq!( power.len(), width * height, "power must have width * height values" ); + let at = |row: usize, col: usize| power[row * width + col]; - let p_max = power.iter().copied().fold(f64::NEG_INFINITY, f64::max); + let p_max = (viewport.row..viewport.row + viewport.height) + .flat_map(|row| (viewport.col..viewport.col + viewport.width).map(move |col| (row, col))) + .map(|(row, col)| at(row, col)) + .fold(f64::NEG_INFINITY, f64::max); if p_max <= 0.0 || config.max_count == 0 { return Vec::new(); } let threshold = p_max * 10f64.powf(-config.range_db / 10.0); - let at = |row: usize, col: usize| power[row * width + col]; + // Relative to the viewport + let (dc, dr) = (viewport.col as f64, viewport.row as f64); + let size = [viewport.width, viewport.height]; let mut candidates: Vec<(usize, usize)> = (0..height) .flat_map(|row| (0..width).map(move |col| (row, col))) @@ -65,21 +123,22 @@ pub fn find_peaks(power: &[f64], [width, height]: [usize; 2], config: &PeakConfi p >= threshold && is_local_max(row, col, p, [width, height], &at) }) .collect(); - candidates.sort_by(|&(r1, c1), &(r2, c2)| at(r2, c2).total_cmp(&at(r1, c1))); + // Strongest first, central ones preferred (see `center_weight_db`) + let weighted = |row: usize, col: usize| { + let penalty = center_penalty_db(col as f64 - dc, row as f64 - dr, size, config); + at(row, col) * 10f64.powf(-penalty / 10.0) + }; + candidates.sort_by(|&(r1, c1), &(r2, c2)| weighted(r2, c2).total_cmp(&weighted(r1, c1))); + let margin = config.edge_margin_cells; + let (max_col, max_row) = ((width - 1) as f64 - margin, (height - 1) as f64 - margin); + // Kept maxima in full grid cells, inside and outside the viewport, for the distance check + let mut kept: Vec<(f64, f64)> = Vec::new(); let mut peaks: Vec = Vec::new(); for (row, col) in candidates { if peaks.len() == config.max_count { break; } - let too_close = peaks.iter().any(|peak| { - let (dr, dc) = (peak.row - row as f64, peak.col - col as f64); - (dr * dr + dc * dc).sqrt() < config.min_dist_cells - }); - if too_close { - continue; - } - let p = at(row, col); // No refinement on the border let col_offset = if col > 0 && col + 1 < width { @@ -92,15 +151,49 @@ pub fn find_peaks(power: &[f64], [width, height]: [usize; 2], config: &PeakConfi } else { 0.0 }; - peaks.push(Peak { - col: col as f64 + col_offset, - row: row as f64 + row_offset, - level_db: 10.0 * (p / p_max).log10(), - }); + let (peak_col, peak_row) = (col as f64 + col_offset, row as f64 + row_offset); + // Before the distance check, so a dropped border maximum does not suppress a peak inside + if peak_col < margin || peak_col > max_col || peak_row < margin || peak_row > max_row { + continue; + } + + let too_close = kept + .iter() + .any(|&(c, r)| (r - peak_row).hypot(c - peak_col) < config.min_dist_cells); + if too_close { + continue; + } + kept.push((peak_col, peak_row)); + + if viewport.contains(peak_col, peak_row) { + peaks.push(Peak { + col: peak_col - dc, + row: peak_row - dr, + level_db: 10.0 * (p / p_max).log10(), + }); + } } peaks } +/// Extra dB a source at (col, row) needs, see [`PeakConfig::center_weight_db`]: +/// `center_weight_db` times the squared distance from the map center in units of half the map +/// width (0 in the center, the full weight at the left and right border, about a third at the +/// top and bottom border of a 16:9 map, more in the corners). +pub fn center_penalty_db( + col: f64, + row: f64, + [width, height]: [usize; 2], + config: &PeakConfig, +) -> f64 { + let half_width = (width as f64 - 1.0) / 2.0; + if half_width <= 0.0 { + return 0.0; + } + let (dc, dr) = (col - half_width, row - (height as f64 - 1.0) / 2.0); + config.center_weight_db * (dc * dc + dr * dr) / (half_width * half_width) +} + /// No 8-neighbor is larger; on ties only the first cell in row-major order wins fn is_local_max( row: usize, @@ -145,7 +238,7 @@ fn parabola_offset(left: f64, center: f64, right: f64) -> f64 { #[cfg(test)] mod test { - use super::{PeakConfig, find_peaks}; + use super::{PeakConfig, Viewport, center_penalty_db, find_peaks, find_peaks_in}; const SIZE: [usize; 2] = [40, 30]; @@ -201,11 +294,13 @@ mod test { (30.0, 25.0, 0.5, 1.0), ]; let power = gaussians(&blobs); + // Without center weighting, so only the levels decide the order let config = |range_db, min_dist_cells, max_count| PeakConfig { - enabled: true, range_db, min_dist_cells, max_count, + center_weight_db: 0.0, + ..PeakConfig::default() }; let cols_rows = |config: PeakConfig| -> Vec<(i64, i64)> { find_peaks(&power, SIZE, &config) @@ -226,13 +321,98 @@ mod test { assert_eq!(cols_rows(config(6.0, 4.0, 1)), [(10, 10)]); } - /// A flat plateau gives one peak, a map without positive values none. + /// A flat plateau gives one peak (its first cell, on the border, so without margin), a map + /// without positive values none. #[test] fn plateau_and_empty_map() { let flat = vec![1.0; SIZE[0] * SIZE[1]]; - assert_eq!(find_peaks(&flat, SIZE, &PeakConfig::default()).len(), 1); + let no_margin = PeakConfig { + edge_margin_cells: 0.0, + ..PeakConfig::default() + }; + assert_eq!(find_peaks(&flat, SIZE, &no_margin).len(), 1); let negative = vec![-1.0; SIZE[0] * SIZE[1]]; assert!(find_peaks(&negative, SIZE, &PeakConfig::default()).is_empty()); } + + /// A source outside the map has its maximum on the border: dropped with `edge_margin_cells` + /// 1. It does not suppress a weaker peak inside within `min_dist_cells`. + #[test] + fn border_maximum_dropped() { + let power = gaussians(&[(-1.0, 15.0, 2.0, 1.5), (3.0, 15.0, 0.8, 1.0)]); + let config = PeakConfig { + edge_margin_cells: 1.0, + ..PeakConfig::default() + }; + let peaks = find_peaks(&power, SIZE, &config); + + assert_eq!(peaks.len(), 1); + assert!((peaks[0].col - 3.0).abs() < 0.5, "col {}", peaks[0].col); + } + + /// A viewport inside the map: a strong source outside it (its lobe reaching in) is not + /// returned but suppresses the maximum it leaves inside within `min_dist_cells`; a peak + /// inside comes out relative to the viewport, below 0 dB (the suppressed border maximum is + /// the visible maximum). + #[test] + fn viewport_hides_outside_peaks() { + let viewport = Viewport { + col: 8, + row: 6, + width: 24, + height: 18, + }; + // Outside at col 5, the lobe leaves a maximum at col 8 (the viewport border, 3 cells + // away), a real source at (20, 15) + let power = gaussians(&[ + (5.0, 15.0, 4.0, 2.0), + (8.0, 15.0, 1.5, 0.5), + (20.0, 15.0, 1.0, 1.5), + ]); + let config = PeakConfig { + center_weight_db: 0.0, + ..PeakConfig::default() + }; + let peaks = find_peaks_in(&power, SIZE, viewport, &config); + + assert_eq!(peaks.len(), 1, "{peaks:?}"); + assert!((peaks[0].col - 12.0).abs() < 0.1, "col {}", peaks[0].col); + assert!((peaks[0].row - 9.0).abs() < 0.1, "row {}", peaks[0].row); + assert!(peaks[0].level_db < 0.0, "level {}", peaks[0].level_db); + + // Without the outside source, the border maximum is a peak of its own + let power = gaussians(&[(8.0, 15.0, 1.5, 0.5), (20.0, 15.0, 1.0, 1.5)]); + let peaks = find_peaks_in(&power, SIZE, viewport, &config); + assert_eq!(peaks.len(), 2, "{peaks:?}"); + assert!((peaks[0].col).abs() < 0.1, "col {}", peaks[0].col); + } + + /// Two equally strong peaks with `max_count` 1: the central one wins. + #[test] + fn central_peak_preferred() { + let power = gaussians(&[(4.0, 15.0, 1.0, 1.0), (20.0, 15.0, 1.0, 1.0)]); + let config = PeakConfig { + max_count: 1, + ..PeakConfig::default() + }; + let peaks = find_peaks(&power, SIZE, &config); + + assert_eq!(peaks.len(), 1); + assert!((peaks[0].col - 20.0).abs() < 1e-6); + } + + /// 0 in the center, the full weight at the left and right border. + #[test] + fn center_penalty() { + let config = PeakConfig::default(); + let [width, height] = SIZE; + let (center_col, center_row) = ((width - 1) as f64 / 2.0, (height - 1) as f64 / 2.0); + assert_eq!( + center_penalty_db(center_col, center_row, SIZE, &config), + 0.0 + ); + let edge = center_penalty_db(0.0, center_row, SIZE, &config); + assert!((edge - config.center_weight_db).abs() < 1e-12); + } } diff --git a/crates/core/src/pipeline.rs b/crates/core/src/pipeline.rs index 486748e..089c9bb 100644 --- a/crates/core/src/pipeline.rs +++ b/crates/core/src/pipeline.rs @@ -1,21 +1,30 @@ //! Processing chain after the `FrameSource`: STFT, covariance, beamformer and map output. use std::{ + collections::VecDeque, io, ops::{ControlFlow, RangeInclusive}, time::Instant, }; +use nalgebra::SVector; use serde::Deserialize; use crate::{ - BoxError, FrameSource, + BoxError, Complex64, FrameSource, dsp::{ - beamformer::{Beamformer, Steering}, + beam_signal::{BeamSignal, TRACK_SAMPLE_RATE, decimation_factor}, + beamformer::{Beamformer, Steering, coherent_and_received_power}, covariance::SlidingCovariance, stft::{Stft, StftStream}, }, - geometry::{camera::Camera, mic_array::MicArray, scan_grid::ScanGrid}, + geometry::{ + camera::Camera, + mic_array::MicArray, + scan_grid::{MAX_OFF_AXIS_DEG, ScanGrid, Viewport}, + }, + peaks::{PeakConfig, center_penalty_db, find_peaks_in}, + tracking::{Track, Tracker, TrackerConfig}, }; /// Processing parameters, e.g. from a config file; the defaults are tested with the UMA-16 at 48 kHz @@ -36,6 +45,18 @@ pub struct PipelineConfig { pub grid_width: u32, /// Focus distance of the scan grid in meters pub grid_z: f64, + /// The scan grid reaches this many degrees beyond the camera field of view on every side (as + /// far as [`MAX_OFF_AXIS_DEG`] allows), at the same cell size; only the visible part is shown. + /// A source outside the image then has its maximum at its own direction instead of on the + /// image border, where the main lobe (about 34 degrees at 3 kHz) would otherwise leak in, and + /// a deconvolving beamformer (CLEAN-SC) takes it out of the visible part. The map time grows + /// with the number of cells (about 3x at 15 degrees with the 83 degree camera). + pub grid_margin_deg: f64, + /// Leave impulsive sounds (snapping, clapping, a hammer) out of the map entirely: STFT + /// frames whose energy jumps `TRANSIENT_DB` above the recent median are not accumulated. + /// Off, they are shown like any other source, but a second, steady map without those + /// frames decides which tracks can be speech (`Track::steady`). + pub skip_transients: bool, } impl Default for PipelineConfig { @@ -48,10 +69,43 @@ impl Default for PipelineConfig { f_hi_hz: 4100.0, grid_width: 64, grid_z: 2.0, + grid_margin_deg: 15.0, + skip_transients: false, } } } +/// Lower end of the upper part of the map band, in Hz: from here on the 12.6 cm array resolves +/// directions (a source 30 degrees off the beam is attenuated), below every beam carries every +/// source. [`Track::source_upper_db`] is measured here. +pub const UPPER_BAND_HZ: f64 = 2000.0; + +/// The noise floor of the source check is learned over this many seconds of maps: per map the +/// largest coherent power over a coarse subgrid (every `FLOOR_STRIDE`th cell), of which the +/// lowest is the floor (minimum statistics), so that the quietest moment sets it: the pauses +/// between words of a talker are enough, whatever happens in the rest of the window. Power, not +/// the fraction of the received power: +/// a steady background source (fan) has a fraction near 1 in silence, so no fraction could be +/// above it, but a talker has more power. With uncorrelated noise the coherent power +/// fluctuates around zero, so a single point (such as the map maximum) is useless in dB; the +/// largest of about 30 points is a stable statistic. +pub const FLOOR_WINDOW_S: f64 = 60.0; +const FLOOR_STRIDE: usize = 8; +/// The power a track's upper-band share is measured against is the received power minus what +/// the other tracks explain, but at least this fraction of it (the share is boosted by at most +/// 10 dB when loud tracks explain almost everything) +const UPPER_UNEXPLAINED_MIN: f64 = 0.1; +/// An STFT frame is a transient if its energy over the map band is this far above the median +/// of the last `TRANSIENT_HISTORY` frames (0.26 s at 48 kHz; at least `TRANSIENT_MIN_HISTORY` +/// of them). Speech varies by less than 10 dB within that time; the first frames of a sound +/// after silence count as transients until the median has caught up (about 0.13 s). +const TRANSIENT_DB: f64 = 10.0; +const TRANSIENT_HISTORY: usize = 24; +const TRANSIENT_MIN_HISTORY: usize = 8; +/// A track is steady if the steady map (without transient frames) has a peak within +/// `peaks.min_dist_cells` of it that is at most this far below that map's maximum +const STEADY_RANGE_DB: f64 = 6.0; + #[derive(Debug, thiserror::Error)] pub enum PipelineError { #[error("invalid pipeline config: {0}")] @@ -82,21 +136,71 @@ impl PipelineConfig { if self.grid_width == 0 || self.grid_z <= 0.0 { return invalid("grid_width must be at least 1 and grid_z > 0"); } + if !(0.0..90.0).contains(&self.grid_margin_deg) { + return invalid("grid_margin_deg must be at least 0 and below 90"); + } Ok(()) } } -/// Everything after the `FrameSource`: STFT, covariance, beamformer and output +/// Everything after the `FrameSource`: STFT, covariance, beamformer, peaks, tracks and output pub struct Pipeline> { stream: StftStream, cov: SlidingCovariance, + /// Like `cov`, without the transient frames; decides which tracks are steady (speech + /// candidates). With a 12.6 cm array a hand snapping next to a talker is inside the beam + /// toward the talker at every speech frequency, so both beams carry the same audio and no + /// audio test can tell the snapping track from the talker; a map without the snaps can. + steady_cov: SlidingCovariance, beamformer: B, - /// Bins summed in the power map + /// Bins summed in the power map, and the ones of them from `UPPER_BAND_HZ` on (all of them + /// if the band ends below) bins: RangeInclusive, + upper_bins: RangeInclusive, + /// Steering of the scan grid, for the source check of peaks and tracks + steering: Steering, + /// Largest coherent power over the subgrid per map (linear, > 0), for the last + /// `floor_window` maps, see `FLOOR_WINDOW_S` + floor_samples: VecDeque, + /// Energy over the map band of the last `TRANSIENT_HISTORY` STFT frames, for the transient + /// check; unused without `skip_transients` + skip_transients: bool, + frame_energies: VecDeque, + floor_window: usize, /// One map every this many STFT frames map_every: usize, - /// Layout of the power map + /// Layout of the power map handed out: the visible part of the scan grid layout: MapLayout, + /// Full scan grid size including the margin, and the visible cells in it + grid_size: [usize; 2], + viewport: Viewport, + peaks: PeakConfig, + tracker: Tracker, + /// Audio toward each track, by track id; empty if the sample rate has no integer + /// decimation factor to `TRACK_SAMPLE_RATE` + beams: Vec<(u64, BeamSignal)>, + track_audio: Option>, +} + +/// What a new track needs to get its beam signal: grid cell -> point -> mic distances +struct TrackAudio { + mic_array: MicArray, + /// Camera at scan grid resolution, see [`ScanGrid::new`] + grid_camera: Camera, + grid_z: f64, + fft_len: usize, + stft_scale: f64, + sample_rate: f64, +} + +impl TrackAudio { + /// Mic distances to the point of grid cell (col, row) at the focus distance + fn distances(&self, track: &Track) -> [f64; M] { + let point = self + .grid_camera + .unproject(track.col, track.row, self.grid_z); + self.mic_array.get_distance_from(&point) + } } #[derive(Clone)] @@ -110,17 +214,28 @@ impl> Pipeline<16, B> { /// UMA-16 pipeline for `sample_rate` in Hz; the scan grid covers the field of view of `camera`. /// /// `make_beamformer` builds the beamformer from the steering of this pipeline's scan grid. + /// Each map's peaks (`peaks`) are followed as tracks (`tracking`), each with the audio from + /// its direction if `sample_rate` is an integer multiple of [`TRACK_SAMPLE_RATE`]. /// - /// Error if `config` is invalid, see [`PipelineConfig::validate`]. + /// Error if `config` or `tracking` is invalid, see [`PipelineConfig::validate`]. pub fn new( sample_rate: f64, config: &PipelineConfig, + peaks: &PeakConfig, + tracking: &TrackerConfig, camera: Camera, make_beamformer: impl FnOnce(Steering<16>) -> B, ) -> Result { config.validate(sample_rate)?; + tracking.validate().map_err(PipelineError::InvalidConfig)?; - let scan_grid = ScanGrid::new(&camera, config.grid_width, config.grid_z); + let scan_grid = ScanGrid::with_margin( + &camera, + config.grid_width, + config.grid_z, + config.grid_margin_deg.to_radians(), + ); + let viewport = scan_grid.viewport(); let n_bins = config.fft_len / 2 + 1; let df = sample_rate / config.fft_len as f64; @@ -128,49 +243,227 @@ impl> Pipeline<16, B> { let bin_lo = (config.f_lo_hz / df).round() as usize; let bin_hi = ((config.f_hi_hz / df).round() as usize).min(n_bins - 1); let bins = bin_lo..=bin_hi; + let bin_upper = (UPPER_BAND_HZ / df).round() as usize; + let upper_bins = if bin_upper > bin_lo && bin_upper <= bin_hi { + bin_upper..=bin_hi + } else { + bins.clone() + }; println!( - "Grid {}x{}, bins {bins:?}, memory {} frames, map every {} frames", + "Grid {}x{} ({}x{} visible, margin {} degrees, at most {MAX_OFF_AXIS_DEG} off axis), \ + bins {bins:?}, memory {} frames, map every {} frames", scan_grid.width(), scan_grid.height(), + viewport.width, + viewport.height, + config.grid_margin_deg, config.memory_frames, config.map_every ); + let stft = Stft::new(config.fft_len); + let track_audio = match decimation_factor(sample_rate) { + Some(_) => Some(TrackAudio { + mic_array: MicArray::uma16(), + grid_camera: Camera { + intrinsics: camera.intrinsics.scaled(config.grid_width), + mount: camera.mount.clone(), + }, + grid_z: config.grid_z, + fft_len: config.fft_len, + stft_scale: stft.scale(), + sample_rate, + }), + None => { + println!( + "No track audio: {sample_rate} Hz is no integer multiple of {TRACK_SAMPLE_RATE} Hz" + ); + None + } + }; + + let steering = Steering::new(&scan_grid, &MicArray::uma16(), sample_rate, config.fft_len); Ok(Self { - stream: StftStream::new(Stft::new(config.fft_len)), + stream: StftStream::new(stft), cov: SlidingCovariance::new(n_bins, config.memory_frames), - beamformer: make_beamformer(Steering::new( - &scan_grid, - &MicArray::uma16(), - sample_rate, - config.fft_len, - )), + steady_cov: SlidingCovariance::new(n_bins, config.memory_frames), + beamformer: make_beamformer(steering.clone()), bins, + upper_bins, + steering, + floor_samples: VecDeque::new(), + skip_transients: config.skip_transients, + frame_energies: VecDeque::new(), + floor_window: (FLOOR_WINDOW_S * sample_rate + / (config.map_every * config.fft_len / 2) as f64) + .ceil() + .max(1.0) as usize, map_every: config.map_every, layout: MapLayout { camera, - grid_width: scan_grid.width(), - grid_height: scan_grid.height(), + grid_width: viewport.width, + grid_height: viewport.height, }, + grid_size: [scan_grid.width(), scan_grid.height()], + viewport, + peaks: peaks.clone(), + tracker: Tracker::new(tracking.clone()), + beams: Vec::new(), + track_audio, }) } } +impl> Pipeline { + /// Index in the full grid of the visible cell nearest to (col, row) in visible cells + fn point(&self, col: f64, row: f64) -> usize { + let vp = self.viewport; + let col = (col.round().max(0.0) as usize).min(vp.width - 1) + vp.col; + let row = (row.round().max(0.0) as usize).min(vp.height - 1) + vp.row; + row * self.grid_size[0] + col + } + + /// Power arriving coherently from `point` over the map band, see + /// [`coherent_and_received_power`] + fn coherent_power(&self, point: usize) -> f64 { + coherent_and_received_power( + self.cov.estimate(), + self.bins.clone(), + &self.steering, + point, + ) + .0 + } + + /// Coherent power from the cell nearest to (col, row) in dB above `floor` + fn above_floor_db(&self, col: f64, row: f64, floor: f64) -> f64 { + 10.0 * (self.coherent_power(self.point(col, row)).max(1e-30) / floor).log10() + } + + /// Noise floor (linear): the lowest of `floor_samples`, infinite without samples + fn floor(&self) -> f64 { + self.floor_samples + .iter() + .copied() + .fold(f64::INFINITY, f64::min) + } + + /// Continue the tracks with the visible peaks of `power` (the full grid), mark the ones + /// with a peak in `steady` (the map without transient frames) as steady, measure each + /// track's coherent power above the noise floor, hand each track the audio of its beam + /// since the previous map, then point the beams at the new track positions. + fn update_tracks(&mut self, power: &[f64], steady: &[f64]) { + let size = [self.layout.grid_width, self.layout.grid_height]; + let peaks = find_peaks_in(power, self.grid_size, self.viewport, &self.peaks); + let steady_config = PeakConfig { + range_db: STEADY_RANGE_DB, + ..self.peaks.clone() + }; + let steady_peaks = find_peaks_in(steady, self.grid_size, self.viewport, &steady_config); + // The current map is part of the floor window; no sample without any coherent power + // (e.g. a source that delivers zeros). Over the full grid: a source outside the image + // is part of the background too. + let [width, height] = self.grid_size; + let sample = (FLOOR_STRIDE / 2..height) + .step_by(FLOOR_STRIDE) + .flat_map(|row| { + let cols = (FLOOR_STRIDE / 2..width).step_by(FLOOR_STRIDE); + cols.map(move |col| row * width + col) + }) + .map(|point| self.coherent_power(point)) + .fold(f64::NEG_INFINITY, f64::max); + if sample > 0.0 { + if self.floor_samples.len() >= self.floor_window { + self.floor_samples.pop_front(); + } + self.floor_samples.push_back(sample); + } + let floor = self.floor(); + self.tracker.update(&peaks); + + let cov = self.cov.estimate(); + let tracks = self.tracker.tracks(); + let above: Vec = tracks + .iter() + .map(|track| self.above_floor_db(track.col, track.row, floor)) + .collect(); + let upper: Vec<(f64, f64)> = tracks + .iter() + .map(|track| { + let point = self.point(track.col, track.row); + coherent_and_received_power(cov, self.upper_bins.clone(), &self.steering, point) + }) + .collect(); + // Mean mic power over the upper bins, the same for every track + let received = upper.first().map_or(0.0, |&(_, received)| received); + let coherent_sum: f64 = upper.iter().map(|&(coherent, _)| coherent.max(0.0)).sum(); + + let min_dist = self.peaks.min_dist_cells; + let tracks = self.tracker.tracks_mut(); + for ((track, above), &(coherent, _)) in tracks.iter_mut().zip(above).zip(&upper) { + track.steady = steady_peaks + .iter() + .any(|peak| (peak.col - track.col).hypot(peak.row - track.row) <= min_dist); + // The track's share of the upper-band power that the other tracks do not explain: + // a loud source next to a talker (snapping) would otherwise hide the talker's share, + // while a talker outside the image, who is no track, still shows as leakage + let others = coherent_sum - coherent.max(0.0); + let unexplained = (received - others).max(received * UPPER_UNEXPLAINED_MIN); + let share = if unexplained > 0.0 { + coherent / unexplained + } else { + 0.0 + }; + track.source_db = above; + track.edge_penalty_db = center_penalty_db(track.col, track.row, size, &self.peaks); + track.source_upper_db = 10.0 * share.max(1e-10).log10(); + } + + self.beams + .retain(|(id, _)| tracks.iter().any(|track| track.id == *id)); + let Some(audio) = &self.track_audio else { + return; + }; + for track in tracks { + let distances = audio.distances(track); + match self.beams.iter_mut().find(|(id, _)| *id == track.id) { + Some((_, beam)) => { + track.samples = beam.take_output(); + beam.steer(&distances); + } + None => { + track.samples = Vec::new(); + let beam = BeamSignal::new( + audio.fft_len, + audio.stft_scale, + audio.sample_rate, + &distances, + ); + self.beams.push((track.id, beam)); + } + } + } + } +} + /// Read `source` through the pipeline; one power map every `map_every` STFT frames. /// /// The covariance is a sliding average over about `memory_frames` frames, so consecutive maps /// overlap. -/// Each map is passed to `on_map` together with its [`MapLayout`]. Stops when the source ends -/// (`Ok(0)`), when `on_map` returns `Break`, or on the first error from `source` or `on_map`. +/// Each map is passed to `on_map` together with its [`MapLayout`] and the current tracks (each +/// with its audio since the previous map). Stops when the source ends (`Ok(0)`), when `on_map` +/// returns `Break`, or on the first error from `source` or `on_map`. pub fn run, B: Beamformer, const M: usize>( source: &mut S, mut pipeline: Pipeline, - mut on_map: impl FnMut(&[f64], &MapLayout) -> Result, BoxError>, + mut on_map: impl FnMut(&[f64], &MapLayout, &[Track]) -> Result, BoxError>, ) -> Result<(), PipelineError> { let mut buf = vec![0.0; 1024 * M]; let mut total_frames = 0; let mut out = vec![0.0; pipeline.beamformer.n_points()]; + let mut steady = vec![0.0; pipeline.beamformer.n_points()]; + let mut visible = vec![0.0; pipeline.layout.grid_width * pipeline.layout.grid_height]; let mut map_index = 0; // STFT frames since the last map let mut new_frames = 0; @@ -182,9 +475,21 @@ pub fn run, B: Beamformer, const M: usize>( } total_frames += frames; - // Closures capture disjoint fields, so `stream` and `cov` can be borrowed together + // Closures capture disjoint fields, so `stream`, `cov` and `beams` can be borrowed + // together. Transients stay out of the steady covariance (and of the map with + // `skip_transients`), not out of the beams (the audio). + let bins = pipeline.bins.clone(); pipeline.stream.push(&buf[..frames * M], |snapshots, _l| { - pipeline.cov.accumulate(snapshots); + let transient = is_transient(&mut pipeline.frame_energies, &snapshots[bins.clone()]); + if !transient { + pipeline.steady_cov.accumulate(snapshots); + } + if !(transient && pipeline.skip_transients) { + pipeline.cov.accumulate(snapshots); + } + for (_, beam) in &mut pipeline.beams { + beam.push_frame(snapshots); + } new_frames += 1; }); @@ -193,6 +498,16 @@ pub fn run, B: Beamformer, const M: usize>( pipeline .beamformer .power_map(pipeline.cov.estimate(), pipeline.bins.clone(), &mut out); + // The steady map is the map itself when the transients are skipped anyway, or + // before a steady frame arrived + if pipeline.skip_transients || pipeline.steady_cov.count() == 0 { + steady.copy_from_slice(&out); + } else { + let cov = pipeline.steady_cov.estimate(); + pipeline + .beamformer + .power_map(cov, pipeline.bins.clone(), &mut steady); + } println!( "map {map_index}: power_map {:.1} ms", start.elapsed().as_secs_f64() * 1e3 @@ -201,7 +516,12 @@ pub fn run, B: Beamformer, const M: usize>( new_frames = 0; map_index += 1; - let flow = on_map(&out, &pipeline.layout).map_err(PipelineError::MapHandler)?; + pipeline.update_tracks(&out, &steady); + pipeline + .viewport + .crop(&out, pipeline.grid_size[0], &mut visible); + let flow = on_map(&visible, &pipeline.layout, pipeline.tracker.tracks()) + .map_err(PipelineError::MapHandler)?; if flow.is_break() { break; } @@ -214,3 +534,561 @@ pub fn run, B: Beamformer, const M: usize>( ); Ok(()) } + +/// Whether the frame with the band snapshots `snapshots` is a transient, see `TRANSIENT_DB`; +/// records its energy in `history` either way +fn is_transient( + history: &mut VecDeque, + snapshots: &[SVector], +) -> bool { + let energy: f64 = snapshots.iter().map(|x| x.norm_squared()).sum(); + let mut sorted: Vec = history.iter().copied().collect(); + sorted.sort_by(f64::total_cmp); + let transient = sorted.len() >= TRANSIENT_MIN_HISTORY + && energy > sorted[sorted.len() / 2] * 10f64.powf(TRANSIENT_DB / 10.0); + if history.len() >= TRANSIENT_HISTORY { + history.pop_front(); + } + history.push_back(energy); + transient +} + +#[cfg(test)] +mod test { + use std::{f64::consts::PI, io, ops::ControlFlow}; + + use super::{Pipeline, PipelineConfig, run}; + use crate::{ + FrameSource, Point3, + dsp::{beam_signal::TRACK_SAMPLE_RATE, beamformer::clean_sc::CleanSc}, + geometry::mic_array::MicArray, + peaks::PeakConfig, + source::simulator::{ + Simulator, + scene::{MicNoiseParams, Scene, Source}, + signal::{Noise, NoiseParams, Signal, Sine}, + }, + test_util::{SAMPLE_RATE, test_camera}, + tracking::{Track, TrackerConfig}, + }; + + /// Amplitude of the component at `freq` (least squares fit of sine and cosine) + fn amplitude_at(signal: &[f32], freq: f64) -> f64 { + let (mut c, mut s) = (0.0, 0.0); + for (n, &y) in signal.iter().enumerate() { + let phase = 2.0 * PI * freq * n as f64 / TRACK_SAMPLE_RATE; + c += y as f64 * phase.cos(); + s += y as f64 * phase.sin(); + } + 2.0 * (c * c + s * s).sqrt() / signal.len() as f64 + } + + /// Mic noise alone at `level_db` for `duration_s` + fn noise_scene(level_db: f64, duration_s: f64, seed: u64) -> Scene { + Scene { + sample_rate: SAMPLE_RATE, + duration_s, + mic_noise_params: MicNoiseParams { seed, level_db }, + sources: Vec::new(), + } + } + + /// `0` until it ends, then `1` + struct Chain(A, B); + + impl, B: FrameSource<16>> FrameSource<16> for Chain { + fn sample_rate(&self) -> f64 { + self.0.sample_rate() + } + + fn read(&mut self, buf: &mut [f64]) -> io::Result { + match self.0.read(buf)? { + 0 => self.1.read(buf), + frames => Ok(frames), + } + } + } + + /// Two sines on the focus plane after two quiet seconds (the noise floor needs a quiet map): + /// two confirmed tracks at the sources, each with audio in which its own source dominates. + /// CLEAN-SC, because the broad DAS main lobes of the two sources overlap and pull both peaks + /// about two cells toward each other. + #[test] + fn tracks_follow_sources_with_their_audio() { + let config = PipelineConfig::default(); + let sources = [ + (Point3::new(0.6, 0.2, config.grid_z), 3000.0), + (Point3::new(-0.6, 0.2, config.grid_z), 3500.0), + ]; + let scene = Scene { + sample_rate: SAMPLE_RATE, + duration_s: 2.0, + mic_noise_params: MicNoiseParams { + seed: 1, + level_db: 20.0, + }, + sources: sources + .iter() + .map(|&(pos, freq)| Source { + pos, + signal: Signal::Sine(Sine::new(1.0, freq, 0.0)), + }) + .collect(), + }; + let quiet = Simulator::new(noise_scene(20.0, 2.0, 3), MicArray::uma16()).unwrap(); + let mut sim = Chain(quiet, Simulator::new(scene, MicArray::uma16()).unwrap()); + let camera = test_camera(); + let pipeline = Pipeline::new( + SAMPLE_RATE, + &config, + &PeakConfig::default(), + &TrackerConfig::default(), + camera.clone(), + |steering| CleanSc::new(steering, 0.9, 10, 20.0, 1.5, true, true), + ) + .unwrap(); + + // Last map's tracks, and all audio per track id + let mut last: Vec = Vec::new(); + let mut audio: Vec<(u64, Vec)> = Vec::new(); + run(&mut sim, pipeline, |_power, layout, tracks| { + for track in tracks { + match audio.iter_mut().find(|(id, _)| *id == track.id) { + Some((_, samples)) => samples.extend_from_slice(&track.samples), + None => audio.push((track.id, track.samples.clone())), + } + } + last = tracks.to_vec(); + assert_eq!(layout.grid_width, config.grid_width as usize); + Ok(ControlFlow::Continue(())) + }) + .unwrap(); + + let min_db = PeakConfig::default().min_above_floor_db; + let confirmed: Vec<&Track> = last + .iter() + .filter(|t| t.confirmed && t.source_db >= min_db) + .collect(); + assert_eq!(confirmed.len(), 2, "tracks: {last:?}"); + + let intrinsics = &camera.intrinsics; + let grid_width = config.grid_width as f64; + let grid_height = (grid_width * intrinsics.height as f64 / intrinsics.width as f64).round(); + for (pos, freq) in sources { + // Source in grid cells, as `Intrinsics::scaled` + let (u, v) = camera.project(&pos); + let col = (u + 0.5) * grid_width / intrinsics.width as f64 - 0.5; + let row = (v + 0.5) * grid_height / intrinsics.height as f64 - 0.5; + let track = confirmed + .iter() + .min_by(|a, b| { + let d = |t: &Track| (t.col - col).hypot(t.row - row); + d(a).total_cmp(&d(b)) + }) + .unwrap(); + let dist = (track.col - col).hypot(track.row - row); + assert!( + dist < 1.0, + "track at ({}, {}), source at ({col}, {row})", + track.col, + track.row + ); + + // Each source gets about half of the received power, far above the quiet maps + let above = track.source_db; + assert!( + above > 10.0, + "track {}: {above} dB above the floor", + track.id + ); + let upper = track.source_upper_db; + assert!( + upper > -5.0, + "track {}: {upper} dB coherent above 2 kHz", + track.id + ); + + let (_, samples) = audio.iter().find(|(id, _)| *id == track.id).unwrap(); + // At least a second of audio + assert!( + samples.len() > TRACK_SAMPLE_RATE as usize, + "{} samples", + samples.len() + ); + let other = sources.iter().find(|(p, _)| *p != pos).unwrap().1; + let own_db = + 20.0 * (amplitude_at(samples, freq) / amplitude_at(samples, other)).log10(); + // About 30 dB and 12 dB: depends on where the other source falls in this beam's + // sidelobe pattern (near a null at 3.5 kHz, not at 3 kHz) + assert!( + own_db > 8.0, + "track {}: own source {own_db:.1} dB above the other", + track.id + ); + } + } + + /// Last visible map and tracks of a loud sine 52 degrees above the camera axis (the test + /// camera sees 39) next to a quieter one inside the image, CLEAN-SC, `margin_deg` of scan + /// grid margin; plus the inside source's grid cell + fn outside_source_maps(margin_deg: f64) -> (Vec, Vec, (f64, f64), [usize; 2]) { + let config = PipelineConfig { + grid_margin_deg: margin_deg, + ..PipelineConfig::default() + }; + let camera = test_camera(); + let inside = Point3::new(0.3, 0.1, config.grid_z); + let intrinsics = &camera.intrinsics; + let outside = camera.unproject( + intrinsics.cx, + -0.3 * intrinsics.height as f64, + config.grid_z, + ); + let scene = Scene { + sample_rate: SAMPLE_RATE, + duration_s: 1.5, + mic_noise_params: MicNoiseParams { + seed: 1, + level_db: 20.0, + }, + sources: vec![ + Source { + pos: inside, + signal: Signal::Sine(Sine::new(1.0, 3000.0, 0.0)), + }, + Source { + pos: outside, + signal: Signal::Sine(Sine::new(3.0, 3500.0, 0.0)), + }, + ], + }; + let mut sim = Simulator::new(scene, MicArray::uma16()).unwrap(); + let pipeline = Pipeline::new( + SAMPLE_RATE, + &config, + &PeakConfig::default(), + &TrackerConfig::default(), + camera.clone(), + |steering| CleanSc::new(steering, 0.9, 10, 20.0, 1.5, true, true), + ) + .unwrap(); + + let mut last_map = Vec::new(); + let mut last_tracks = Vec::new(); + let mut size = [0, 0]; + run(&mut sim, pipeline, |power, layout, tracks| { + last_map = power.to_vec(); + last_tracks = tracks.to_vec(); + size = [layout.grid_width, layout.grid_height]; + Ok(ControlFlow::Continue(())) + }) + .unwrap(); + + // Source in grid cells, as `Intrinsics::scaled` + let (u, v) = camera.project(&inside); + let col = (u + 0.5) * size[0] as f64 / intrinsics.width as f64 - 0.5; + let row = (v + 0.5) * size[1] as f64 / intrinsics.height as f64 - 0.5; + (last_map, last_tracks, (col, row), size) + } + + /// Largest value of the top row relative to the map maximum, in dB + fn top_row_db(map: &[f64], [width, _]: [usize; 2]) -> f64 { + let max = |values: &[f64]| values.iter().copied().fold(f64::NEG_INFINITY, f64::max); + 10.0 * (max(&map[..width]) / max(map)).log10() + } + + /// With the scan grid margin, CLEAN-SC puts a source above the image on a margin cell: the + /// visible map peaks at the source inside, the top row stays dark and the only confirmed + /// track is the inside source. Without the margin the outside source is cleaned onto the + /// top border, where its main lobe reaches in. + #[test] + fn source_outside_the_image_stays_out() { + let (map, tracks, (col, row), size) = outside_source_maps(15.0); + let argmax = (0..map.len()) + .max_by(|&a, &b| map[a].total_cmp(&map[b])) + .unwrap(); + let (max_col, max_row) = ((argmax % size[0]) as f64, (argmax / size[0]) as f64); + assert!( + (max_col - col).hypot(max_row - row) <= 1.0, + "maximum at ({max_col}, {max_row}), inside source at ({col}, {row})" + ); + let top = top_row_db(&map, size); + assert!(top < -15.0, "top row {top} dB with margin"); + let confirmed: Vec<&Track> = tracks.iter().filter(|t| t.confirmed).collect(); + assert!(!confirmed.is_empty(), "{tracks:?}"); + for track in confirmed { + let dist = (track.col - col).hypot(track.row - row); + assert!( + dist <= 1.5, + "track {track:?}, inside source at ({col}, {row})" + ); + } + + let (map, _, _, size) = outside_source_maps(0.0); + let top = top_row_db(&map, size); + assert!(top > -6.0, "top row {top} dB without margin"); + } + + /// Mic noise alone, loud: every map has a maximum, but the coherent power from it stays at + /// the noise floor, so no track would be shown. + #[test] + fn noise_alone_gives_no_tracks() { + let mut sim = Simulator::new(noise_scene(80.0, 6.0, 2), MicArray::uma16()).unwrap(); + let pipeline = Pipeline::new( + SAMPLE_RATE, + &PipelineConfig::default(), + &PeakConfig::default(), + &TrackerConfig::default(), + test_camera(), + |steering| CleanSc::new(steering, 0.9, 10, 20.0, 1.5, true, true), + ) + .unwrap(); + + let min_db = PeakConfig::default().min_above_floor_db; + let mut maps = 0; + run(&mut sim, pipeline, |_power, _layout, tracks| { + assert!( + tracks.iter().all(|t| t.source_db < min_db), + "map {maps}: {tracks:?}" + ); + maps += 1; + Ok(ControlFlow::Continue(())) + }) + .unwrap(); + assert!(maps > 50, "{maps} maps"); + } + + /// `talker` plus `hand` snapping: the hand's sound times a decaying envelope (8 ms) every + /// 300 ms, the same for all mics (the inter-mic delays are far shorter) + struct Snapping { + talker: Simulator<16>, + hand: Simulator<16>, + hand_buf: Vec, + n: u64, + } + + impl FrameSource<16> for Snapping { + fn sample_rate(&self) -> f64 { + self.talker.sample_rate() + } + + fn read(&mut self, buf: &mut [f64]) -> io::Result { + let frames = self.talker.read(buf)?; + self.hand_buf.resize(frames * 16, 0.0); + let hand_frames = self.hand.read(&mut self.hand_buf)?; + for frame in 0..hand_frames { + let t = (self.n + frame as u64) as f64 / SAMPLE_RATE; + let envelope = (-(t % 0.3) / 0.008).exp(); + for m in 0..16 { + buf[frame * 16 + m] += envelope * self.hand_buf[frame * 16 + m]; + } + } + self.n += frames as u64; + Ok(frames) + } + } + + /// Snapping (loud band noise in 8 ms bursts) next to a talker: with `skip_transients` the + /// snaps stay out of the map, only the talker is a track; without, the hand is one too, + /// but not a steady one (inside the beam toward the talker its audio is the talker's: no + /// way to tell them apart by sound). + #[test] + fn transients_are_not_steady() { + let scene = |pos, signal| Scene { + sample_rate: SAMPLE_RATE, + duration_s: 3.0, + mic_noise_params: MicNoiseParams { + seed: 8, + level_db: 40.0, + }, + sources: vec![Source { pos, signal }], + }; + let talker = scene( + Point3::new(0.6, 0.2, 2.0), + Signal::Sine(Sine::new(1.0, 3000.0, 0.0)), + ); + let hand = scene( + Point3::new(-0.2, 0.2, 2.0), + Signal::Noise(Noise::from(NoiseParams { + seed: 11, + n_components: 200, + f_lo_hz: 1000.0, + f_hi_hz: 6000.0, + level_db: 110.0, + })), + ); + let min_db = PeakConfig::default().min_above_floor_db; + + let mut shown_cols = Vec::new(); + for skip_transients in [true, false] { + let config = PipelineConfig { + skip_transients, + ..PipelineConfig::default() + }; + let quiet = Simulator::new(noise_scene(40.0, 1.0, 9), MicArray::uma16()).unwrap(); + let snapping = Snapping { + talker: Simulator::new(talker.clone(), MicArray::uma16()).unwrap(), + hand: Simulator::new(hand.clone(), MicArray::uma16()).unwrap(), + hand_buf: Vec::new(), + n: 0, + }; + let mut sim = Chain(quiet, snapping); + let pipeline = Pipeline::new( + SAMPLE_RATE, + &config, + &PeakConfig::default(), + &TrackerConfig::default(), + test_camera(), + |steering| CleanSc::new(steering, 0.9, 10, 20.0, 1.5, true, true), + ) + .unwrap(); + let mut last = Vec::new(); + run(&mut sim, pipeline, |_power, _layout, tracks| { + last = tracks.to_vec(); + Ok(ControlFlow::Continue(())) + }) + .unwrap(); + let mut shown: Vec<(f64, bool)> = last + .iter() + .filter(|t| t.confirmed && t.source_db >= min_db) + .map(|t| (t.col, t.steady)) + .collect(); + shown.sort_by(|a, b| a.0.total_cmp(&b.0)); + shown_cols.push(shown); + } + + // Talker at about column 25, hand at about 34 (+x is left in the image) + let [with_skip, without] = [&shown_cols[0], &shown_cols[1]]; + assert_eq!(with_skip.len(), 1, "with skip: {with_skip:?}"); + assert!( + with_skip[0].0 < 28.0 && with_skip[0].1, + "with skip: {with_skip:?}" + ); + assert_eq!(without.len(), 2, "without: {without:?}"); + assert!(without[0].0 < 28.0 && without[0].1, "talker: {without:?}"); + assert!(without[1].0 > 30.0 && !without[1].1, "hand: {without:?}"); + } + + /// A talker next to a source 10 dB louder in the same band (snapping): both are tracks, + /// and the talker's upper-band share is measured against the power the snapping does not + /// explain, so it stays near 0 dB (about -10 dB against all received power). + #[test] + fn louder_neighbor_does_not_hide_talker() { + // 0.8 m apart (9 grid cells): closer, the clean beam merges the two spots + let talker = Point3::new(0.6, 0.2, 2.0); + let snapping = Point3::new(-0.2, 0.2, 2.0); + let scene = Scene { + sample_rate: SAMPLE_RATE, + duration_s: 2.0, + mic_noise_params: MicNoiseParams { + seed: 6, + level_db: 40.0, + }, + sources: vec![ + Source { + pos: talker, + signal: Signal::Sine(Sine::new(1.0, 3000.0, 0.0)), + }, + Source { + pos: snapping, + signal: Signal::Sine(Sine::new(3.2, 3500.0, 0.0)), + }, + ], + }; + let quiet = Simulator::new(noise_scene(40.0, 1.0, 7), MicArray::uma16()).unwrap(); + let mut sim = Chain(quiet, Simulator::new(scene, MicArray::uma16()).unwrap()); + let pipeline = Pipeline::new( + SAMPLE_RATE, + &PipelineConfig::default(), + &PeakConfig::default(), + &TrackerConfig::default(), + test_camera(), + |steering| CleanSc::new(steering, 0.9, 10, 20.0, 1.5, true, true), + ) + .unwrap(); + + let mut last = Vec::new(); + run(&mut sim, pipeline, |_power, _layout, tracks| { + last = tracks.to_vec(); + Ok(ControlFlow::Continue(())) + }) + .unwrap(); + // The talker at x = 0.6 is left of the snapping at x = -0.2 (+x is left in the image) + let confirmed: Vec<&Track> = last.iter().filter(|t| t.confirmed).collect(); + assert_eq!(confirmed.len(), 2, "{} confirmed tracks", confirmed.len()); + let talker_track = confirmed + .iter() + .min_by(|a, b| a.col.total_cmp(&b.col)) + .unwrap(); + assert!( + talker_track.col < 28.0, + "talker track at col {}", + talker_track.col + ); + let upper = talker_track.source_upper_db; + assert!(upper > -4.0, "talker's upper-band share {upper} dB"); + } + + /// A steady source from the start is the noise floor (no dot: a fan in a real room), a + /// source 14 dB louder that appears later gets a track. + #[test] + fn steady_background_is_the_floor() { + let fan = Point3::new(-0.5, -0.3, 2.0); + let talker = Point3::new(0.6, 0.2, 2.0); + let source = |pos, amplitude| Source { + pos, + signal: Signal::Sine(Sine::new(amplitude, 3000.0, 0.0)), + }; + let scene = |sources: Vec, seed| Scene { + sample_rate: SAMPLE_RATE, + duration_s: 2.0, + mic_noise_params: MicNoiseParams { + seed, + level_db: 40.0, + }, + sources, + }; + let only_fan = Simulator::new(scene(vec![source(fan, 0.2)], 4), MicArray::uma16()).unwrap(); + let both = scene(vec![source(fan, 0.2), source(talker, 1.0)], 5); + let mut sim = Chain(only_fan, Simulator::new(both, MicArray::uma16()).unwrap()); + let pipeline = Pipeline::new( + SAMPLE_RATE, + &PipelineConfig::default(), + &PeakConfig::default(), + &TrackerConfig::default(), + test_camera(), + |steering| CleanSc::new(steering, 0.9, 10, 20.0, 1.5, true, true), + ) + .unwrap(); + + let min_db = PeakConfig::default().min_above_floor_db; + let (mut maps, mut confirmed) = (0, Vec::new()); + run(&mut sim, pipeline, |_power, _layout, tracks| { + // The first second is the fan alone (18 maps); then the talker starts + if maps < 17 { + assert!( + tracks.iter().all(|t| t.source_db < min_db), + "map {maps}: {tracks:?}" + ); + } + confirmed = tracks + .iter() + .filter(|t| t.confirmed && t.source_db >= min_db) + .cloned() + .collect(); + maps += 1; + Ok(ControlFlow::Continue(())) + }) + .unwrap(); + assert_eq!(confirmed.len(), 1, "{confirmed:?}"); + let track = &confirmed[0]; + // +x is left in the image (`Mount` turns the camera 180 degrees about z): the talker at + // x = 0.6 is at about column 25, the fan at -0.5 at about 38 + assert!(track.col < 30.0, "track at col {}", track.col); + assert!( + track.source_db > 10.0, + "{} dB above the floor", + track.source_db + ); + } +} diff --git a/crates/core/src/render.rs b/crates/core/src/render.rs index 5f2269b..ce2b523 100644 --- a/crates/core/src/render.rs +++ b/crates/core/src/render.rs @@ -5,24 +5,29 @@ use image::{ imageops::{self, FilterType}, }; -use crate::{Point3, peaks::Peak, pipeline::MapLayout}; +use crate::{Point3, pipeline::MapLayout, tracking::Track}; /// Marker half length in camera pixels const MARKER_HALF_LEN: i64 = 10; -/// Radius of the dot per estimated source in camera pixels +/// Radius of the dot per confirmed track in camera pixels const PEAK_RADIUS: i64 = 8; +/// Dot colors: track, and track classified as speech +const TRACK_RGB: Rgb = Rgb([40, 120, 255]); +const SPEECH_RGB: Rgb = Rgb([63, 185, 80]); -/// Power map as image at camera resolution, with a red cross per true source and a blue dot per -/// peak. +/// Power map as image at camera resolution, with a red cross per true source and a dot per +/// confirmed track: blue, or green if its id is in `speech`. /// /// - `power`: row-major like the scan grid, `grid_width * grid_height` values /// - `src_pos`: true source positions, array frame -/// - `peaks`: estimated sources in grid cells, see [`crate::peaks::find_peaks`] +/// - `tracks`: estimated sources in grid cells, see [`crate::tracking::Tracker`] +/// - `speech`: ids of the tracks classified as speech pub fn map_image( power: &[f64], layout: &MapLayout, src_pos: &[Point3], - peaks: &[Peak], + tracks: &[Track], + speech: &[u64], ) -> RgbImage { // One grid pixel per map value; x = col, y = row, same as `power[row * w + col]` let w = layout.grid_width; @@ -43,16 +48,21 @@ pub fn map_image( // Grid cell centers to camera pixel centers, as `Intrinsics::scaled` let s_x = intrinsics.width as f64 / layout.grid_width as f64; let s_y = intrinsics.height as f64 / layout.grid_height as f64; - for peak in peaks { - let u = ((peak.col + 0.5) * s_x - 0.5).round() as i64; - let v = ((peak.row + 0.5) * s_y - 0.5).round() as i64; + for track in tracks.iter().filter(|track| track.confirmed) { + let u = ((track.col + 0.5) * s_x - 0.5).round() as i64; + let v = ((track.row + 0.5) * s_y - 0.5).round() as i64; + let color = if speech.contains(&track.id) { + SPEECH_RGB + } else { + TRACK_RGB + }; for dy in -PEAK_RADIUS..=PEAK_RADIUS { for dx in -PEAK_RADIUS..=PEAK_RADIUS { let (x, y) = (u + dx, v + dy); let inside = (0..big.width() as i64).contains(&x) && (0..big.height() as i64).contains(&y); if dx * dx + dy * dy <= PEAK_RADIUS * PEAK_RADIUS && inside { - big.put_pixel(x as u32, y as u32, Rgb([40, 120, 255])); + big.put_pixel(x as u32, y as u32, color); } } } diff --git a/crates/core/src/source/simulator/reference.rs b/crates/core/src/source/simulator/reference.rs index 5417412..28ccf35 100644 --- a/crates/core/src/source/simulator/reference.rs +++ b/crates/core/src/source/simulator/reference.rs @@ -51,7 +51,8 @@ pub fn reference_covariance( let bin_lower_range = bin.saturating_sub(BINS_AROUND); let bin_upper_range = cmp::min(fft_len / 2, bin + BINS_AROUND); - for k in bin_lower_range..=bin_upper_range { + let bins = bin_lower_range..=bin_upper_range; + for (k, r_k) in bins.clone().zip(&mut r_ref[bins]) { // distance to bin k, in cycles per sample let offset = component.freq_hz / f_s - k as f64 / fft_len as f64; @@ -62,7 +63,7 @@ pub fn reference_covariance( .sum(); let s_q = (component.amplitude / 2.0).powi(2) * w_k.norm_sqr(); - r_ref[k] += a_ah.scale(s_q); + *r_k += a_ah.scale(s_q); } } } diff --git a/crates/core/src/source/video/avfoundation.rs b/crates/core/src/source/video/avfoundation.rs index 95a3fdc..cb9d0ec 100644 --- a/crates/core/src/source/video/avfoundation.rs +++ b/crates/core/src/source/video/avfoundation.rs @@ -32,8 +32,9 @@ use objc2_core_media::{CMSampleBuffer, CMTime, CMVideoFormatDescriptionGetDimens use objc2_core_video::{ CVPixelBufferGetBaseAddress, CVPixelBufferGetBytesPerRow, CVPixelBufferGetHeight, CVPixelBufferGetPixelFormatType, CVPixelBufferGetWidth, CVPixelBufferLockBaseAddress, - CVPixelBufferLockFlags, CVPixelBufferUnlockBaseAddress, kCVPixelBufferPixelFormatTypeKey, - kCVPixelFormatType_32BGRA, kCVReturnSuccess, + CVPixelBufferLockFlags, CVPixelBufferUnlockBaseAddress, kCVPixelBufferHeightKey, + kCVPixelBufferPixelFormatTypeKey, kCVPixelBufferWidthKey, kCVPixelFormatType_32BGRA, + kCVReturnSuccess, }; use objc2_foundation::{ NSArray, NSDictionary, NSError, NSNumber, NSObject, NSObjectProtocol, NSString, @@ -44,6 +45,9 @@ use crate::{ImageSource, VideoFrame}; /// `next_frame` gives up after this long without a frame (e.g. camera unplugged) const FRAME_TIMEOUT: Duration = Duration::from_secs(2); +/// Formats that cannot reach this frame rate (or the requested one, if lower) are taken only if +/// nothing else fits +const MIN_LIVE_FPS: f64 = 25.0; /// Request camera access; required once before any other camera call (macOS asks the user on /// the first run). Returns immediately, see [`is_authorized`]. @@ -148,28 +152,36 @@ impl CameraSource { a.distance(width, height, fps), b.distance(width, height, fps), ); - a.0.cmp(&b.0).then(a.1.total_cmp(&b.1)) + a.0.cmp(&b.0).then(a.1.cmp(&b.1)).then(a.2.total_cmp(&b.2)) }) .ok_or_else(|| VideoError::Camera("camera lists no video formats".to_string()))?; - if (option.width, option.height) != (width as i32, height as i32) { - eprintln!( - "camera {index}: requested {width}x{height}, got {}x{}", - option.width, option.height - ); - } + let (min_fps, max_fps) = option.fps_range(); + eprintln!( + "camera {index}: {}x{} at {min_fps:.0}-{max_fps:.0} fps, {} (requested \ + {width}x{height} at {fps:.0} fps)", + option.width, + option.height, + fourcc(&option.format) + ); let input = unsafe { AVCaptureDeviceInput::deviceInputWithDevice_error(&device) } .map_err(ns_error)?; - // 32BGRA: AVFoundation converts from the device format, no decoding on our side + // 32BGRA at the format's own size: AVFoundation converts from the device format, no + // decoding on our side. Without the size it may scale the output down. let output = unsafe { AVCaptureVideoDataOutput::new() }; - // SAFETY: CFString and NSString are toll-free bridged - let key: &NSString = unsafe { - &*(kCVPixelBufferPixelFormatTypeKey as *const CFString).cast::() - }; let pixel_format = NSNumber::numberWithUnsignedInt(kCVPixelFormatType_32BGRA); - let value: &AnyObject = &pixel_format; - let settings = NSDictionary::from_slices(&[key], &[value]); + let output_width = NSNumber::numberWithInt(option.width); + let output_height = NSNumber::numberWithInt(option.height); + let keys = unsafe { + [ + cf_key(kCVPixelBufferPixelFormatTypeKey), + cf_key(kCVPixelBufferWidthKey), + cf_key(kCVPixelBufferHeightKey), + ] + }; + let values: [&AnyObject; 3] = [&pixel_format, &output_width, &output_height]; + let settings = NSDictionary::from_slices(&keys, &values); unsafe { output.setVideoSettings(Some(&settings)); output.setAlwaysDiscardsLateVideoFrames(true); @@ -198,7 +210,8 @@ impl CameraSource { session.addOutput(&output); // Setting the active format switches the session to the input-priority preset, - // so the session keeps it instead of choosing its own + // so the session keeps it instead of choosing its own. Never set that preset + // explicitly: `setSessionPreset` throws for it (uncaught, aborts the process). if let Err(e) = device.lockForConfiguration() { session.commitConfiguration(); return Err(ns_error(e)); @@ -296,14 +309,16 @@ unsafe fn to_rgba(sample_buffer: &CMSampleBuffer) -> Option { let stride = CVPixelBufferGetBytesPerRow(&buffer); let base = CVPixelBufferGetBaseAddress(&buffer).cast::(); - let mut pixels = Vec::with_capacity(width * height * 4); + let mut pixels = vec![0; width * height * 4]; if !base.is_null() { - for row in 0..height { + for (row, out) in pixels.chunks_exact_mut(width * 4).enumerate() { // SAFETY: the locked buffer holds `height` rows of `stride` bytes, each starting // with `width` BGRA pixels let line = unsafe { slice::from_raw_parts(base.add(row * stride), width * 4) }; - for bgra in line.chunks_exact(4) { - pixels.extend_from_slice(&[bgra[2], bgra[1], bgra[0], 255]); + let (rgba, _) = out.as_chunks_mut::<4>(); + let (bgra, _) = line.as_chunks::<4>(); + for (rgba, bgra) in rgba.iter_mut().zip(bgra) { + *rgba = [bgra[2], bgra[1], bgra[0], 255]; } } } @@ -329,13 +344,16 @@ impl FormatOption { unsafe { (self.range.minFrameRate(), self.range.maxFrameRate()) } } - /// Sort key against the request: resolution mismatch in pixels, then how far `fps` lies + /// Sort key against the request, smaller is better: too slow for a live image (cannot + /// reach `fps` or `MIN_LIVE_FPS`, whichever is lower; a camera may list 1920x1080 also + /// uncompressed at 5 fps), then the resolution mismatch in pixels, then how far `fps` lies /// outside the frame rate range - fn distance(&self, width: u32, height: u32, fps: f64) -> (u64, f64) { + fn distance(&self, width: u32, height: u32, fps: f64) -> (bool, u64, f64) { let pixels = self.width.abs_diff(width as i32) + self.height.abs_diff(height as i32); let (min_fps, max_fps) = self.fps_range(); + let too_slow = max_fps < fps.min(MIN_LIVE_FPS); let fps_off = (min_fps - fps).max(fps - max_fps).max(0.0); - (u64::from(pixels), fps_off) + (too_slow, u64::from(pixels), fps_off) } /// Frame duration for `fps`, clamped to the range; the range's own durations at its ends @@ -395,6 +413,14 @@ fn fourcc(format: &AVCaptureDeviceFormat) -> String { code.to_be_bytes().iter().map(|&b| b as char).collect() } +/// A CoreVideo key as `NSString` (toll-free bridged) +/// +/// # Safety +/// `key` must be a valid CFString constant. +unsafe fn cf_key(key: &'static CFString) -> &'static NSString { + unsafe { &*(key as *const CFString).cast::() } +} + fn video_media_type() -> &'static NSString { unsafe { AVMediaTypeVideo }.expect("AVMediaTypeVideo exists on macOS") } diff --git a/crates/core/src/tracking.rs b/crates/core/src/tracking.rs new file mode 100644 index 0000000..f2723bf --- /dev/null +++ b/crates/core/src/tracking.rs @@ -0,0 +1,342 @@ +//! Peaks followed from map to map: stable ids, smoothed positions and the audio of each source. + +use serde::Deserialize; + +use crate::peaks::{Peak, PeakConfig}; + +/// Tracker parameters, e.g. from a config file +#[derive(Clone, Debug, PartialEq, Deserialize)] +#[serde(default, deny_unknown_fields)] +pub struct TrackerConfig { + /// A peak continues a track if it is at most this far away, in grid cells. Small enough that + /// the maxima of noise, which jump around, do not keep continuing one track (4 cells is + /// about 5 degrees, a person walking at 2 m moves about 2.5 cells per map) + pub max_dist_cells: f64, + /// A track counts as confirmed after a matching peak in `min_hits` of the last + /// `confirm_window` maps (not necessarily in a row: a source at the detection limit comes and + /// goes; noise maxima do not stay in one place long enough). A snap stays in the covariance + /// for about `memory_frames`, i.e. 4 maps, and is confirmed. + pub min_hits: usize, + pub confirm_window: usize, + /// Maps in a row without a matching peak until a track is dropped + pub max_missed: usize, + /// Weight of the new peak in the smoothed position, in (0, 1]; 1 follows the peak exactly + /// (less puts the dot between two spots when the map's maximum jumps, e.g. to a reflection) + pub smoothing: f64, +} + +impl Default for TrackerConfig { + fn default() -> Self { + Self { + max_dist_cells: 4.0, + min_hits: 3, + confirm_window: 5, + max_missed: 5, + smoothing: 1.0, + } + } +} + +impl TrackerConfig { + /// Error message if a value cannot work + pub fn validate(&self) -> Result<(), String> { + if self.max_dist_cells <= 0.0 || self.min_hits == 0 { + return Err("tracking: max_dist_cells must be > 0 and min_hits >= 1".to_string()); + } + if !(self.min_hits..=MAX_CONFIRM_WINDOW).contains(&self.confirm_window) { + return Err(format!( + "tracking: confirm_window must be in min_hits..={MAX_CONFIRM_WINDOW}" + )); + } + if !(self.smoothing > 0.0 && self.smoothing <= 1.0) { + return Err("tracking: smoothing must be in (0, 1]".to_string()); + } + Ok(()) + } +} + +/// `confirm_window` is kept as bits of a `u64` +const MAX_CONFIRM_WINDOW: usize = 64; + +/// A source followed over several maps +#[derive(Clone, Debug)] +pub struct Track { + /// Unique within one pipeline run, never reused + pub id: u64, + /// Smoothed position in grid cells, like [`Peak`] + pub col: f64, + pub row: f64, + /// Level of the last matching peak relative to the map maximum, in dB + pub level_db: f64, + /// Filled by the pipeline: the power arriving coherently from the track over the map band, + /// in dB above the noise floor (see [`crate::peaks::PeakConfig::min_above_floor_db`]), and + /// its share of the power received over the band's upper part (see + /// [`crate::pipeline::UPPER_BAND_HZ`]) that the other tracks do not explain, in dB + pub source_db: f64, + pub source_upper_db: f64, + /// Filled by the pipeline: how much more `source_db` a track needs at its position to be + /// shown, see [`crate::peaks::center_penalty_db`] + pub edge_penalty_db: f64, + /// Also a peak of the map without transient frames (see + /// [`crate::pipeline::PipelineConfig::skip_transients`]): a continuous sound, not snapping + /// or clapping; filled by the pipeline. Only steady tracks can be speech. + pub steady: bool, + /// Matching peaks since the track started + pub hits: usize, + /// One bit per map, newest in bit 0: 1 if a peak matched + pub recent: u64, + /// Maps in a row without a matching peak (0: seen in the current map) + pub missed: usize, + /// Seen in `min_hits` of `confirm_window` maps; stays confirmed until dropped + pub confirmed: bool, + /// Delay-and-sum audio toward the track since the previous map, mono at + /// [`crate::dsp::beam_signal::TRACK_SAMPLE_RATE`]; filled by the pipeline + pub samples: Vec, +} + +impl Track { + /// Whether to show (and record) the track: confirmed, `min_above_floor_db` (plus its + /// `edge_penalty_db`) above the noise floor and at least `min_coherence_db` coherent above + /// 2 kHz, see [`PeakConfig`] + pub fn shown(&self, peaks: &PeakConfig) -> bool { + self.confirmed + && self.source_db - self.edge_penalty_db >= peaks.min_above_floor_db + && self.source_upper_db >= peaks.min_coherence_db + } +} + +/// Matches the peaks of each map to the tracks of the previous one +pub struct Tracker { + config: TrackerConfig, + tracks: Vec, + next_id: u64, +} + +impl Tracker { + pub fn new(config: TrackerConfig) -> Self { + Self { + config, + tracks: Vec::new(), + next_id: 0, + } + } + + /// Current tracks, oldest first + pub fn tracks(&self) -> &[Track] { + &self.tracks + } + + pub fn tracks_mut(&mut self) -> &mut [Track] { + &mut self.tracks + } + + /// Continue the tracks with the peaks of a new map. + /// + /// Greedy nearest neighbor: the closest (track, peak) pair within `max_dist_cells` is matched + /// first, then the next closest among the rest. Unmatched peaks start new tracks; unmatched + /// tracks count a miss and are dropped after more than `max_missed`. + pub fn update(&mut self, peaks: &[Peak]) { + for track in &mut self.tracks { + track.recent <<= 1; + } + let window_mask = u64::MAX >> (64 - self.config.confirm_window); + + let max_dist = self.config.max_dist_cells; + let mut pairs: Vec<(f64, usize, usize)> = self + .tracks + .iter() + .enumerate() + .flat_map(|(t, track)| { + peaks.iter().enumerate().filter_map(move |(p, peak)| { + let dist = (track.col - peak.col).hypot(track.row - peak.row); + (dist <= max_dist).then_some((dist, t, p)) + }) + }) + .collect(); + pairs.sort_by(|a, b| a.0.total_cmp(&b.0)); + + let mut track_matched = vec![false; self.tracks.len()]; + let mut peak_matched = vec![false; peaks.len()]; + for (_, t, p) in pairs { + if track_matched[t] || peak_matched[p] { + continue; + } + track_matched[t] = true; + peak_matched[p] = true; + + let (track, peak) = (&mut self.tracks[t], &peaks[p]); + let alpha = self.config.smoothing; + track.col += alpha * (peak.col - track.col); + track.row += alpha * (peak.row - track.row); + track.level_db = peak.level_db; + track.hits += 1; + track.recent |= 1; + track.missed = 0; + let recent_hits = (track.recent & window_mask).count_ones() as usize; + track.confirmed |= recent_hits >= self.config.min_hits; + } + + for (track, matched) in self.tracks.iter_mut().zip(&track_matched) { + if !matched { + track.missed += 1; + } + } + let max_missed = self.config.max_missed; + self.tracks.retain(|track| track.missed <= max_missed); + + for (peak, _) in peaks + .iter() + .zip(&peak_matched) + .filter(|(_, matched)| !**matched) + { + self.tracks.push(Track { + id: self.next_id, + col: peak.col, + row: peak.row, + level_db: peak.level_db, + source_db: 0.0, + source_upper_db: 0.0, + edge_penalty_db: 0.0, + steady: false, + hits: 1, + recent: 1, + missed: 0, + confirmed: self.config.min_hits <= 1, + samples: Vec::new(), + }); + self.next_id += 1; + } + } +} + +#[cfg(test)] +mod test { + use super::{Tracker, TrackerConfig}; + use crate::peaks::Peak; + + fn peak(col: f64, row: f64) -> Peak { + Peak { + col, + row, + level_db: 0.0, + } + } + + fn ids(tracker: &Tracker) -> Vec { + tracker.tracks().iter().map(|t| t.id).collect() + } + + /// Tracker with position smoothing 0.5 + fn smoothed() -> Tracker { + Tracker::new(TrackerConfig { + smoothing: 0.5, + ..TrackerConfig::default() + }) + } + + /// A moving peak keeps its id; its position follows with the smoothing weight. + #[test] + fn moving_peak_keeps_id() { + let mut tracker = smoothed(); + tracker.update(&[peak(10.0, 10.0)]); + tracker.update(&[peak(12.0, 10.0)]); + tracker.update(&[peak(14.0, 11.0)]); + + assert_eq!(ids(&tracker), [0]); + let track = &tracker.tracks()[0]; + // 10 -> 11 -> 12.5, 10 -> 10 -> 10.5 with smoothing 0.5 + assert_eq!((track.col, track.row), (12.5, 10.5)); + assert_eq!(track.hits, 3); + } + + /// A peak farther than `max_dist_cells` starts a new track; the old one counts a miss. + #[test] + fn jump_starts_new_track() { + let mut tracker = Tracker::new(TrackerConfig::default()); + tracker.update(&[peak(10.0, 10.0)]); + tracker.update(&[peak(30.0, 10.0)]); + + assert_eq!(ids(&tracker), [0, 1]); + assert_eq!(tracker.tracks()[0].missed, 1); + assert_eq!(tracker.tracks()[1].missed, 0); + } + + /// Confirmed after `min_hits` matching maps, also with misses in between, stays confirmed + /// through misses, dropped after more than `max_missed`. + #[test] + fn confirmation_and_drop() { + let config = TrackerConfig::default(); + let mut tracker = Tracker::new(config.clone()); + for _ in 0..config.min_hits - 1 { + tracker.update(&[peak(5.0, 5.0)]); + tracker.update(&[]); + } + assert!(!tracker.tracks()[0].confirmed); + tracker.update(&[peak(5.0, 5.0)]); + assert!(tracker.tracks()[0].confirmed); + + for _ in 0..config.max_missed { + tracker.update(&[]); + } + assert!( + tracker.tracks()[0].confirmed, + "kept while missed <= max_missed" + ); + tracker.update(&[]); + assert!(tracker.tracks().is_empty()); + } + + /// A peak in only 2 of every 5 maps (noise at one spot now and then) is never confirmed; the + /// same spot in 3 of 5 is. + #[test] + fn confirmation_needs_hits_within_window() { + let mut tracker = Tracker::new(TrackerConfig::default()); + for map in 0..20 { + let hit = map % 5 == 0 || map % 5 == 2; + let peaks = if hit { + vec![peak(5.0, 5.0)] + } else { + Vec::new() + }; + tracker.update(&peaks); + } + assert!(tracker.tracks().iter().all(|track| !track.confirmed)); + + let mut tracker = Tracker::new(TrackerConfig::default()); + for map in 0..5 { + let hit = map % 2 == 0; + let peaks = if hit { + vec![peak(5.0, 5.0)] + } else { + Vec::new() + }; + tracker.update(&peaks); + } + assert!(tracker.tracks()[0].confirmed); + } + + /// Maxima that jump by more than the gate every map (noise) start new tracks and none is + /// ever confirmed. + #[test] + fn jumping_peaks_never_confirmed() { + let mut tracker = Tracker::new(TrackerConfig::default()); + for map in 0..30 { + let col = 5.0 + (map % 6) as f64 * 5.0; + tracker.update(&[peak(col, 10.0)]); + } + assert!(tracker.tracks().iter().all(|track| !track.confirmed)); + } + + /// Two close tracks and two peaks: each peak goes to its nearest track, not both to one. + #[test] + fn nearest_pairs_first() { + let mut tracker = smoothed(); + tracker.update(&[peak(10.0, 10.0), peak(14.0, 10.0)]); + // Both new peaks are within the gate of both tracks + tracker.update(&[peak(13.0, 10.0), peak(11.0, 10.0)]); + + assert_eq!(ids(&tracker), [0, 1]); + let cols: Vec = tracker.tracks().iter().map(|t| t.col).collect(); + assert_eq!(cols, [10.5, 13.5]); + } +} diff --git a/model/speech_detector.json b/model/speech_detector.json new file mode 100644 index 0000000..bbd675f --- /dev/null +++ b/model/speech_detector.json @@ -0,0 +1 @@ +{"schema_version": 1, "feature_count": 62, "sample_rate": 16000, "hop_length": 160, "feature_contract": "pyprak_scripts/features.py: 40 MFCC + 12 delta + 6 delta2 + 4 spectral", "layers": [{"kind": "affine", "scale": [0.00903458520770073, 0.02680492401123047, 0.042081814259290695, 0.05415978282690048, 0.06465879082679749, 0.07288204878568649, 0.07900968939065933, 0.08886684477329254, 0.0990627333521843, 0.10773312300443649, 0.11015938222408295, 0.11045488715171814, 0.11624784022569656, 0.1267399936914444, 0.1209503710269928, 0.13257014751434326, 0.1428944170475006, 0.14655201137065887, 0.14732584357261658, 0.14931920170783997, 0.15122033655643463, 0.1602208912372589, 0.1606220304965973, 0.1753852367401123, 0.1730862408876419, 0.1852727085351944, 0.18956641852855682, 0.20086956024169922, 0.20433656871318817, 0.21986274421215057, 0.22539737820625305, 0.2346411794424057, 0.24079792201519012, 0.2529231309890747, 0.2482237070798874, 0.27608489990234375, 0.27056071162223816, 0.2957681715488434, 0.3043786287307739, 0.313010573387146, 0.11357281357049942, 0.2701909840106964, 0.38047561049461365, 0.5079962015151978, 0.5834354162216187, 0.6027722358703613, 0.682680070400238, 0.7432363629341125, 0.8039057850837708, 0.8277605175971985, 0.8866171836853027, 0.905602216720581, 0.2922310531139374, 0.5677617788314819, 0.7705474495887756, 0.9861716628074646, 1.0818071365356445, 1.0936806201934814, 0.0022905117366462946, 0.0011472164187580347, 0.0006232150481082499, 8.216388702392578], "bias": [4.006008148193359, -1.7166410684585571, 0.35222792625427246, -0.4020303785800934, 0.7814828753471375, 0.23014266788959503, 0.8701620101928711, 0.2684904932975769, 0.6871669292449951, 0.3514683246612549, 0.6913602948188782, 0.4005829095840454, 0.8378274440765381, 0.3912826180458069, 0.8174170255661011, 0.37434741854667664, 0.6441043615341187, 0.2459273636341095, 0.45908722281455994, 0.056539181619882584, 0.20747263729572296, -0.05391579866409302, 0.09356880187988281, 0.009641745127737522, 0.17420198023319244, 0.010596156120300293, 0.22780469059944153, 0.06606791913509369, 0.3559349477291107, 0.09595699608325958, 0.22077582776546478, 0.13345924019813538, 0.33053654432296753, 0.1275435984134674, 0.17454014718532562, -0.03972994536161423, 0.18397493660449982, -0.07418519258499146, 0.10934960097074509, -0.031099524348974228, -0.06686248630285263, 0.00813467986881733, 0.02366945706307888, 0.014885038137435913, 0.022591982036828995, 0.00890407059341669, 0.016380617395043373, 0.006187969818711281, 0.01527105551213026, 0.010509436950087547, 0.012499476782977581, 0.003489679889753461, 0.05347999185323715, -0.0054405550472438335, -0.01608923450112343, -0.010050147771835327, -0.013537704013288021, -0.005708193872123957, -3.87650465965271, -2.170572519302368, -2.2822206020355225, -1.3454694747924805]}, {"kind": "affine", "scale": [1.0275746583938599, 1.021172046661377, 1.0320212841033936, 1.0174366235733032, 1.0354593992233276, 1.0007730722427368, 1.0068320035934448, 1.0267395973205566, 1.0274176597595215, 0.9795882105827332, 1.0061386823654175, 0.9874475002288818, 0.9932839274406433, 0.9402679204940796, 0.9990858435630798, 0.9876692295074463, 0.9580089449882507, 0.948650598526001, 0.9820218086242676, 0.9550267457962036, 0.9464356899261475, 0.9717878699302673, 0.9808120727539062, 0.9649617671966553, 0.9826210141181946, 0.9375925064086914, 0.9732712507247925, 0.9731869697570801, 0.9298090934753418, 0.9391568303108215, 0.9465303421020508, 0.9571443796157837, 0.9361108541488647, 0.9424995183944702, 0.9419186115264893, 0.9245865345001221, 0.921699583530426, 0.8965137004852295, 0.9181717038154602, 0.920729398727417, 1.0713614225387573, 1.1605061292648315, 1.0793870687484741, 1.119700312614441, 1.0986310243606567, 1.1070648431777954, 1.1169980764389038, 1.0758602619171143, 1.0725821256637573, 0.9524694681167603, 0.9564961791038513, 1.0233073234558105, 1.018093466758728, 1.083428978919983, 0.9998782873153687, 1.0246624946594238, 0.9578417539596558, 1.0000282526016235, 1.0110771656036377, 0.9997307062149048, 0.9995877146720886, 1.0209473371505737], "bias": [-0.047173719853162766, -0.02973002940416336, 0.01999419368803501, 0.021681200712919235, 0.061649393290281296, -0.03424207493662834, 0.009329170919954777, -0.07550649344921112, -0.0027645453810691833, -0.018224243074655533, -0.010307321324944496, -0.014770908281207085, -0.017687764018774033, 0.01901213265955448, -0.023358657956123352, -0.042335402220487595, -0.006433596834540367, -0.049153588712215424, -0.010064369067549706, 0.0054709832184016705, -0.016649309545755386, 0.02218666672706604, -0.027424534782767296, 0.022130992263555527, -0.022863540798425674, 0.0021073585376143456, -0.014680997468531132, -0.01593543402850628, -0.0009279642254114151, -0.030799483880400658, -0.028792638331651688, 0.052536651492118835, -0.03108438104391098, -0.05078846216201782, -0.005928282625973225, -0.016030505299568176, 0.020259574055671692, 0.02346627227962017, -0.03776600584387779, 0.02635575458407402, -0.015065779909491539, 0.019827811047434807, 0.0030660168267786503, -0.04046017676591873, -0.02736121416091919, -0.020898032933473587, 0.02900772914290428, 0.016022412106394768, 0.010633288882672787, -0.03776581212878227, -0.0480029359459877, 0.024898668751120567, 0.025334252044558525, -0.003052423009648919, -0.006040391977876425, 0.013382283970713615, -0.03393726050853729, 0.016349639743566513, 0.05443994328379631, 0.0024620071053504944, -0.007553321775048971, -0.009582496248185635]}, {"kind": "linear", "input": 62, "output": 128, "weight": [0.12957221269607544, 0.06013535335659981, -0.08720818907022476, 0.021517813205718994, -0.01010711956769228, 0.09471358358860016, -0.07964790612459183, -0.06979762762784958, 0.0603373721241951, 0.07981456071138382, -0.034050166606903076, -0.07952748239040375, -0.07468634098768234, -0.008563652634620667, 0.016027232632040977, 0.10745412111282349, -0.023144574835896492, 0.08388998359441757, -0.05330461636185646, 0.07548985630273819, -0.0662480965256691, -0.03895677998661995, 0.06308115273714066, -0.040827538818120956, -0.022327758371829987, -0.057867731899023056, -0.09254015982151031, 0.09779706597328186, -0.030763983726501465, 0.06979367136955261, 0.015186965465545654, -0.051312342286109924, 0.068477563560009, 0.017094796523451805, 0.011979078873991966, 0.0762229636311531, 0.004101841244846582, 0.04262212663888931, 0.07426274567842484, -0.08606614917516708, -0.08240800350904465, -0.09877091646194458, -0.012826751917600632, -0.08022631704807281, 0.058469802141189575, -0.09976410865783691, -0.043597180396318436, -0.10130965709686279, -0.013102523051202297, -0.03428920358419418, -0.004199071787297726, -0.15153087675571442, -0.02645287662744522, 0.06492763012647629, -0.10466339439153671, -0.11184249073266983, -0.08351778984069824, 0.11661344021558762, -0.004083625506609678, 0.1306505650281906, 0.03673518821597099, 0.08188315480947495, 0.05502719804644585, -0.06288230419158936, -0.07507813721895218, -0.06490931659936905, -0.0048418049700558186, -0.011182733811438084, 0.10181083530187607, 0.06886811554431915, 0.07421751320362091, -0.023994024842977524, -0.05548669397830963, -0.037050019949674606, -0.010346116498112679, -0.01167185977101326, -0.1600761115550995, 0.11844325065612793, 0.11899852752685547, 0.03992791846394539, 0.02387526072561741, -0.0784156322479248, 0.044829465448856354, 0.14367513358592987, -0.15520954132080078, 0.015298682264983654, 0.09680530428886414, 0.09594453126192093, -0.06960045546293259, 0.11871292442083359, -0.03265225887298584, 0.008135141804814339, -0.07836056500673294, -0.11399005353450775, -0.004292687401175499, -0.11787916719913483, 0.03108808770775795, -0.0648903027176857, -0.07752881199121475, -0.10722623765468597, 0.06986212730407715, 0.03280060365796089, -0.0981064960360527, 0.06374730169773102, -0.0534704327583313, 0.01341449748724699, -0.039257220923900604, -0.01984921656548977, -0.013141597621142864, 0.09990371018648148, -0.07333371043205261, -0.15168018639087677, -0.07045917958021164, 0.02231205441057682, 0.03298305347561836, 0.05175585299730301, 0.07816214114427567, 0.09297085553407669, -0.009486592374742031, 0.09147224575281143, 0.08596239238977432, 0.04844064265489578, 0.058607108891010284, -0.1093636080622673, -0.05679060146212578, -0.035637494176626205, 0.04809285327792168, -0.10800977051258087, 0.11029309779405594, -0.025778314098715782, -0.08488179743289948, -0.007780883461236954, 0.0912756621837616, 0.057398173958063126, -0.012931911274790764, 0.0900237113237381, 0.12249472737312317, 0.1316387802362442, 0.008467720821499825, 0.0672568827867508, -0.05708972364664078, 0.08153603971004486, -0.10876204073429108, -0.03910187631845474, -0.0946759358048439, -0.013089182786643505, 0.059555888175964355, -0.006975982338190079, 0.001870656618848443, -0.008793672546744347, -0.13206352293491364, -0.06714190542697906, 0.1615893393754959, 0.004653098993003368, -0.06446512043476105, 0.1020650863647461, -0.07421320676803589, -0.0019294826779514551, -0.0963173508644104, -0.03174251317977905, -0.02421160787343979, 0.1356314867734909, -0.014888920821249485, -0.025499098002910614, -0.1362377107143402, 0.09108728170394897, 0.06523867696523666, 0.03493740037083626, -0.04039539769291878, 0.10166440159082413, 0.11179407685995102, 0.08857370167970657, -0.002596505917608738, -0.04517868533730507, 0.004913992714136839, 0.07338234782218933, -0.02378106862306595, -0.05628586187958717, 0.045817941427230835, 0.05878603830933571, -0.049915943294763565, 0.057848963886499405, -0.05109187215566635, 0.13135546445846558, -0.028969325125217438, 0.04183563217520714, 0.13015353679656982, 0.010558836162090302, -0.04663967713713646, -0.03714866563677788, 0.04009249433875084, -0.10688083618879318, 0.04487961158156395, 0.06490790098905563, 0.0273896437138319, -0.02218068391084671, -0.08992435038089752, 0.1004919484257698, 0.1238088384270668, 0.04585923254489899, 0.04673142731189728, -0.0009861175203695893, 0.07788705080747604, -0.09246782958507538, 0.04108225554227829, -0.0952046811580658, 0.15105225145816803, -0.0018750566523522139, 0.02002757601439953, 0.016891377046704292, -0.06265435367822647, 0.039972107857465744, -0.025219706818461418, 0.060019783675670624, -0.06457719951868057, -0.010566215962171555, 0.0018810662440955639, 0.047756627202034, 0.1256483793258667, -0.09122627973556519, 0.04291117191314697, -0.06095227971673012, -0.02453680895268917, 0.07085659354925156, 0.07951676845550537, 0.0062700859270989895, -0.07967136055231094, -0.03834516927599907, 0.06139114499092102, -0.01089246105402708, -0.14558158814907074, 0.08601704239845276, -0.04183327779173851, 0.1272030472755432, 0.09926110506057739, 0.06962728500366211, 0.01970323547720909, -0.005308930296450853, 0.001297307200729847, -0.10394521057605743, 0.10636688023805618, -0.02463626302778721, -0.04336640611290932, 0.09101919829845428, -0.02350529097020626, 0.04863976687192917, -0.0039016699884086847, 0.1023302748799324, 0.10370209068059921, 0.020898528397083282, -0.06850068271160126, -0.11976690590381622, 0.03343725949525833, -0.031754668802022934, -0.012542764656245708, -0.049382057040929794, -0.03348058834671974, -0.0002204942429671064, -0.04572400078177452, -0.03016369417309761, -0.0146709606051445, -0.09714705497026443, -0.15467603504657745, -0.05343740060925484, -0.01865483820438385, 0.078830786049366, -0.1130942553281784, 0.03643554449081421, 0.06858498603105545, 0.11198485642671585, -0.047763146460056305, -0.07332378625869751, 0.08667822927236557, -0.006981587968766689, 0.04933858662843704, 0.08142895251512527, 0.04002254456281662, -0.08639218658208847, -0.09540430456399918, -0.08840985596179962, -0.020537927746772766, 0.09041621536016464, 0.0826677605509758, 0.02634083479642868, -0.03944018483161926, -0.0576302707195282, 0.0726739764213562, 0.061724819242954254, -0.12240460515022278, 0.06422466039657593, 0.07122427225112915, -0.1096511036157608, -0.026395058259367943, 0.09250283241271973, 0.059117238968610764, -0.11580833047628403, 0.10852735489606857, 0.06396643817424774, 0.004852144978940487, 0.02306978590786457, 0.027899935841560364, -0.0911552831530571, 0.034626252949237823, -0.08715944737195969, 0.12191630154848099, -0.0355210080742836, -0.09395845979452133, -0.057397011667490005, -0.05339162051677704, 0.09419247508049011, 0.11328233778476715, -0.1168648898601532, 0.11745265871286392, -0.06231648474931717, -0.06929557025432587, 0.11173523962497711, 0.13453252613544464, 0.10694627463817596, -0.1259225457906723, 0.03944191709160805, 0.005979680921882391, 0.13476833701133728, -0.06053697317838669, -0.12338109314441681, 0.07154673337936401, 0.04418379068374634, 0.08745140582323074, -0.06766945868730545, -0.06329552084207535, -0.027683479711413383, 0.12563611567020416, -0.06849857419729233, -0.07550138235092163, 0.0381152369081974, -0.059885356575250626, -0.07723315805196762, 0.03223111480474472, 0.10037338733673096, -0.11201020330190659, 0.08843892812728882, 0.10608827322721481, -0.08172667026519775, 0.034239716827869415, -0.131149560213089, 0.13443337380886078, 0.035722531378269196, -0.10090568661689758, -0.0035647957120090723, -0.14488007128238678, 0.05244400352239609, 0.05352279916405678, 0.029852617532014847, 0.11712196469306946, 0.02076069824397564, 0.09865661710500717, 0.1274036020040512, -0.039564553648233414, 0.09044402837753296, -0.0021477004047483206, 0.04620435833930969, -0.07699190825223923, -0.08430612087249756, 0.10968881100416183, 0.10247811675071716, 0.09993281215429306, -0.05443474277853966, -0.030885476619005203, -0.07772370427846909, -0.13227638602256775, 0.1455390453338623, -0.0426652617752552, -0.1392165869474411, 0.017921745777130127, 0.13395993411540985, -0.12064078450202942, 0.09305733442306519, 0.07159225642681122, -0.01756453886628151, 0.058650001883506775, 0.055796459317207336, -0.1437104195356369, -0.08458984643220901, 0.12374626845121384, 0.021857786923646927, 0.18540100753307343, 0.013482545502483845, -0.11472193151712418, -0.009760050103068352, -0.04898892343044281, 0.11283785849809647, -0.08991830050945282, -0.03539539873600006, -0.029867378994822502, 0.17521525919437408, -0.11119607090950012, 0.18126845359802246, -0.06866622716188431, -0.05574481934309006, 0.1099361926317215, -0.08221899718046188, -0.08269169181585312, -0.05279035121202469, -0.0015983760822564363, 0.05429304018616676, -0.033444926142692566, 0.0684949979186058, -0.08396917581558228, -0.04458310455083847, -0.07427115738391876, 0.1154041737318039, -0.10407156497240067, -0.07400397956371307, 0.09488105773925781, -0.060738783329725266, 0.10342074930667877, -0.005399164743721485, -0.01219146978110075, 0.006598206236958504, -0.008007748983800411, -0.08651597052812576, -0.03438170626759529, -0.007029884960502386, 0.07918582856655121, -0.05247068777680397, -0.0396844819188118, -0.06574028730392456, 0.01482444629073143, -0.052751556038856506, 0.05422761291265488, -0.09339187294244766, 0.09054957330226898, 0.08119210600852966, 0.09131370484828949, -0.01198622491210699, -0.07452207803726196, 0.1452142596244812, 0.12910230457782745, 0.06390106678009033, -0.015372595749795437, -0.10777632147073746, -0.009262876585125923, -0.058570411056280136, -0.044142529368400574, -0.043852876871824265, 0.0770564153790474, -0.017739711329340935, -0.03139609098434448, 0.09153641015291214, 0.003341846400871873, -0.13472586870193481, -0.06269488483667374, -0.06908829510211945, 0.023855060338974, -0.07222520560026169, 0.046352121978998184, 0.06368667632341385, 0.03543393313884735, 0.06587078422307968, 0.09233757108449936, 0.04248395189642906, -0.07875567674636841, 0.04811473563313484, -0.1013399064540863, -0.09020541608333588, 0.0906343162059784, 0.0958501324057579, 0.03922602906823158, 0.03592991828918457, 0.020249364897608757, -0.05523516610264778, -0.10203360766172409, -0.05840042605996132, -0.04757017642259598, -0.013509081676602364, 0.10429178178310394, 0.08292265236377716, -0.024808622896671295, 0.031222263351082802, -0.12107125669717789, -0.06157563626766205, -0.08365041017532349, 0.07156864553689957, -0.08901167660951614, -0.06467677652835846, 0.014223328791558743, 0.06711269170045853, -0.06417857855558395, 0.05694088712334633, 0.031077196821570396, -0.11464334279298782, 0.15064840018749237, -0.024587206542491913, 0.0472104474902153, -0.03348837420344353, -0.1639070063829422, -0.07515548169612885, -0.1082155629992485, 0.16828657686710358, 0.041609663516283035, -0.09886709600687027, 0.06563226878643036, 0.09657585620880127, 0.05559523031115532, 0.012373565696179867, 0.043965648859739304, 0.014515080489218235, -0.1848951280117035, -0.018105417490005493, 0.035806432366371155, 0.07783842086791992, 0.08631116896867752, 0.0376177541911602, -0.04489367455244064, 0.0804174542427063, 0.1342063695192337, -0.03990517184138298, -0.01611258275806904, 0.10905291140079498, 0.016307584941387177, -0.04649229347705841, 0.01830541528761387, -0.02885882370173931, 0.010098017752170563, -0.10810327529907227, -0.08049240708351135, -0.06449127197265625, 0.027025815099477768, -0.013336296193301678, -0.025533951818943024, -0.011089698411524296, -0.05667174234986305, -0.11253272742033005, 0.0773700699210167, 0.12052946537733078, 0.12481865286827087, 0.14456652104854584, -0.024328405037522316, 0.09079968184232712, -0.009618053212761879, 0.0935032069683075, 0.040070127695798874, 0.03983211889863014, -0.10187023133039474, 0.04993681237101555, -0.026620741933584213, -0.04391098394989967, -0.013683510944247246, 0.05221777409315109, 0.14738979935646057, -0.11080893874168396, 0.007446493487805128, 0.0480719730257988, 0.005550102796405554, -0.06460984796285629, -0.02294054441154003, -0.011845899745821953, -0.10859525203704834, 0.0606578029692173, 0.0391603522002697, 0.049145493656396866, 0.13615509867668152, -0.0972377136349678, 0.050556838512420654, -0.09853141754865646, -0.09594805538654327, -0.05631602555513382, 0.08697497844696045, -0.07767500728368759, -0.12843506038188934, 0.0864139050245285, 0.06318092346191406, -0.10894215106964111, -0.0025103031657636166, -0.0643252283334732, -0.07601518929004669, 0.14826473593711853, -0.045295294374227524, -0.01859218440949917, -0.12009157985448837, 0.04357883334159851, -0.16596147418022156, -0.02031431905925274, 0.022944236174225807, 0.04418326914310455, 0.09329037368297577, 0.05618489533662796, -0.12812183797359467, -0.015336367301642895, 0.05676092952489853, -0.08062240481376648, 0.09604764729738235, 0.014184684492647648, 0.035484421998262405, 0.020160702988505363, 0.046187855303287506, 0.10641610622406006, 0.13578644394874573, 0.09498025476932526, -0.13267101347446442, 0.0644032210111618, -0.042826421558856964, -0.12764424085617065, 0.05070681497454643, 0.06588643789291382, -0.1574045866727829, 0.1322198063135147, 0.08808799833059311, -0.06378009915351868, -0.0950109139084816, -0.08031918853521347, 0.1303468495607376, -0.0857604593038559, -0.08918844908475876, -0.047748301178216934, -0.007397930137813091, -0.031088748946785927, 0.10032948106527328, 0.09302712976932526, 0.07441100478172302, 0.04453517496585846, -0.13521909713745117, 0.03308418020606041, 0.030825115740299225, 0.06892893463373184, 0.015178216621279716, -0.05288992077112198, -0.0018739565275609493, 0.035933613777160645, -0.08745664358139038, 0.006241708528250456, 0.01855400763452053, -0.026899050921201706, -0.0024529893416911364, 0.08410314470529556, -0.12631912529468536, 0.13194283843040466, 0.008971050381660461, -0.11380824446678162, 0.04553523659706116, -0.13250328600406647, -0.010888071730732918, -0.10405244678258896, -0.018960563465952873, -0.03186870366334915, -0.08344268798828125, 0.026584748178720474, 0.06911550462245941, 0.024832917377352715, -0.027013378217816353, -0.16406992077827454, -0.05093236640095711, -0.020999226719141006, 0.13157892227172852, 0.08836738765239716, -0.011576392687857151, -0.03651748225092888, 0.03481396287679672, -0.061143793165683746, -0.07096345722675323, -0.13656717538833618, -0.009856073185801506, -0.05443466454744339, 0.137608140707016, 0.07840414345264435, -0.08396786451339722, 0.037943579256534576, -0.010273570194840431, -0.05404675006866455, 0.06176786124706268, -0.01665312424302101, 0.09495678544044495, -0.029610583558678627, -0.01858268678188324, 0.0750613808631897, 0.03366632014513016, -0.055393170565366745, 0.10946612060070038, -0.13454598188400269, -0.0545470230281353, 0.047963522374629974, -0.07962030917406082, -0.010288211517035961, -0.05158299580216408, 0.08658824115991592, 0.05184130370616913, 0.025979243218898773, -0.13902446627616882, 0.008715476840734482, 0.08186785131692886, 0.014354313723742962, 0.09039706736803055, -0.036549221724271774, 0.008959835395216942, -0.055754028260707855, 0.06010940298438072, 0.001085968455299735, -0.01701570302248001, -0.14115509390830994, -0.10441776365041733, -0.0233742818236351, 0.14592772722244263, 0.001150924013927579, -0.07661154866218567, 0.06726820021867752, -0.02472316101193428, 0.05451656132936478, 0.0936172604560852, 0.08394164592027664, -0.08729325979948044, -1.183204585686326e-05, 0.04046886786818504, 0.08167798072099686, -0.005016962997615337, 0.1420629322528839, -0.0525713786482811, 0.09537585824728012, 0.06863003224134445, 0.010662906803190708, -0.03486943617463112, 0.03686435520648956, -0.1009901762008667, 0.04074499383568764, -0.04302746802568436, -0.13002516329288483, -0.09006606787443161, -0.053172942250967026, 0.036223702132701874, -0.01922369934618473, -0.05820790305733681, -0.019612357020378113, -0.16594962775707245, 0.09290240705013275, -0.11515586823225021, -0.06779292970895767, -0.014581806026399136, -0.11483994126319885, 0.06815017014741898, 0.03897717967629433, 0.011257505044341087, -0.0014229945372790098, 0.03794614225625992, -0.021961061283946037, -0.08626150339841843, -0.04922189936041832, -0.07110822200775146, -0.0521865040063858, 0.05769661068916321, -0.027852173894643784, -0.004215366672724485, 0.0362427644431591, -0.014585796743631363, 0.02271837182343006, -0.007388467434793711, -0.043385040014982224, -0.034198444336652756, 0.09203404188156128, 0.1005992591381073, -0.06998955458402634, -0.09088755398988724, -0.04788908734917641, -0.09812185168266296, -0.02756788395345211, 0.03354422003030777, -0.029616478830575943, -0.10742687433958054, 0.061176933348178864, -0.14965622127056122, -0.1230114996433258, -0.0833517462015152, -0.12193027883768082, 0.09227675199508667, 0.08948581665754318, -0.05884327366948128, 0.1140706017613411, 0.08983954042196274, -0.021623365581035614, 0.010649578645825386, -0.1179991215467453, -0.0043164631351828575, -0.13749374449253082, 0.08512356877326965, 0.04442940279841423, -0.030283497646450996, -0.06253959983587265, 0.00967140682041645, -0.08401631563901901, 0.021931089460849762, 0.01672709546983242, -0.12830635905265808, 0.030753904953598976, 0.006248614750802517, 0.04013608768582344, -0.07656017690896988, -0.03694404661655426, 0.10056820511817932, -0.055682048201560974, -0.027431562542915344, -0.0010151030728593469, 0.029850758612155914, 0.11743495613336563, -0.017821049317717552, 0.08127304911613464, 0.06480193138122559, 0.0631190687417984, -0.06817736476659775, -0.05667820945382118, 0.018127942457795143, 0.09646778553724289, -0.040497105568647385, -0.08779839426279068, 0.09295427054166794, -0.11871602386236191, 0.053858209401369095, 0.09601946175098419, -0.06737122684717178, 0.07777218520641327, 0.03037957288324833, -0.1435280293226242, -0.026345867663621902, 0.11154120415449142, -0.03865163028240204, 0.0355839766561985, -0.052581995725631714, 0.01657748781144619, -0.10155601054430008, 0.005455224309116602, 0.033436231315135956, -0.10664910078048706, 0.03198401257395744, -0.11472169309854507, 0.0668233335018158, 0.0466315895318985, -0.10237772017717361, 0.017833523452281952, 0.09512021392583847, 0.12466121464967728, 0.06592659652233124, 0.09664176404476166, -0.10223466157913208, -0.041425611823797226, 0.07932895421981812, 0.018056504428386688, -0.0337836816906929, 0.13033677637577057, -0.04641956835985184, 0.0737680122256279, 0.11906381696462631, 0.052277982234954834, -0.11969544738531113, -0.013148702681064606, 0.10356996953487396, 0.010265139862895012, 0.06198866292834282, -0.019464615732431412, 0.07740695029497147, -0.03129996359348297, -0.07775021344423294, -0.005834633484482765, -0.055728692561388016, -0.04427308589220047, -0.0005555488169193268, 0.10302983969449997, -0.08185779303312302, 0.00652799429371953, -0.03962574899196625, -0.030474001541733742, 0.003731548087671399, -0.10883841663599014, -0.07951546460390091, -0.03633435070514679, 0.06536786258220673, 0.104426808655262, 0.07017926126718521, -0.05377662181854248, -0.1266954094171524, -0.06651311367750168, 0.06881774216890335, 0.12299743294715881, -0.04549260437488556, -0.03872149810194969, 0.014386588707566261, -0.08271864801645279, 0.074690081179142, 0.10255665332078934, 0.01662859506905079, -0.13972194492816925, -0.018918044865131378, -0.040800683200359344, -0.0003339362738188356, 0.003951923921704292, 0.047238413244485855, 0.14189842343330383, 0.08836285769939423, 0.009693673811852932, -0.10996226221323013, 0.08606842160224915, 0.06649372726678848, -0.13245487213134766, 0.031719036400318146, 0.07660866528749466, -0.1402263045310974, 0.13751491904258728, -0.11357210576534271, 0.08687397837638855, 0.11323346197605133, -0.11296655982732773, 0.06187225505709648, -0.05600837245583534, 0.027147090062499046, 0.04941530153155327, -0.06041751429438591, 0.04980210214853287, -0.10614921897649765, 0.10382477194070816, -0.07384729385375977, -0.025171542540192604, -0.020280851051211357, 0.0028052511624991894, 0.11186222732067108, 0.10169968754053116, -0.1233416497707367, -0.08111519366502762, -0.02057632803916931, -0.030687304213643074, 0.0786968544125557, -0.06402429938316345, 0.040414903312921524, 0.13771282136440277, 0.02532993070781231, -0.07950440049171448, 0.04339924454689026, 0.01496579498052597, -0.10190300643444061, -0.024654807522892952, 0.010342464782297611, 0.1372293084859848, 0.08879978954792023, 0.09864815324544907, -0.028124380856752396, -0.07422178238630295, -0.14653566479682922, 0.07303503900766373, -0.03676484152674675, 0.046907536685466766, 0.08460673689842224, 0.05104617774486542, 0.12154306471347809, -0.04101220890879631, -0.04828871041536331, 0.04036756977438927, 0.12529130280017853, -0.06046685203909874, -0.021125441417098045, -0.02549588866531849, -0.08552224934101105, -0.012674911879003048, 0.08168245851993561, -0.15135252475738525, 0.08004726469516754, -0.008431512862443924, 0.08337697386741638, 0.03805043548345566, 0.0675429254770279, -0.033568598330020905, 0.019424570724368095, -0.03442348539829254, 0.03179148584604263, 0.09335144609212875, -0.11709736287593842, 0.1423959732055664, -0.13374720513820648, 0.00990616250783205, 0.12817402184009552, 0.12171892076730728, -0.10793733596801758, -0.005893400404602289, 0.12148456275463104, 0.007145916111767292, -0.07176163047552109, -0.01017139945179224, 0.14216208457946777, -0.0893649309873581, 0.018098479136824608, 0.04773195460438728, 0.08104991912841797, 0.09883908182382584, -0.049197521060705185, -0.00476080272346735, 0.13957829773426056, 0.08176266402006149, -0.07792213559150696, 0.14034515619277954, 0.11259395629167557, -0.14428497850894928, -0.0403369665145874, 0.023233139887452126, -0.022805802524089813, 0.05151304602622986, -0.06521377712488174, -0.08702606707811356, 0.01928405649960041, 0.08793679624795914, -0.08621250092983246, -0.041808269917964935, -0.07472705096006393, 0.06106976047158241, -0.0034287385642528534, 0.03674691915512085, -0.025437330827116966, 0.0012441773433238268, 0.0025758729316294193, -0.1371784806251526, 0.013349180109798908, -0.034705955535173416, 0.017840148881077766, -0.0421193391084671, -0.05416896194219589, 0.029560737311840057, 0.011179961264133453, 0.09489210695028305, -0.0006223515374585986, -0.014397434890270233, -0.0339672788977623, -0.12609277665615082, 0.1074676662683487, 0.09439941495656967, -0.09359774738550186, -0.03848060593008995, 0.053510140627622604, 0.0038209029007703066, 0.04822593927383423, -0.002912994707003236, 0.13486754894256592, -0.08970264345407486, -0.13885332643985748, 0.013434024527668953, -0.0318455770611763, 0.05426323413848877, 0.059934068471193314, 0.012880168855190277, 0.041575364768505096, 0.09807749092578888, 0.12046369910240173, 0.02715272828936577, -0.07002922892570496, 0.036605220288038254, -0.039047569036483765, -0.10466337203979492, -0.1167759820818901, -0.06567446142435074, -0.04531509429216385, 0.07682529091835022, 0.11960650980472565, -0.06621876358985901, 0.13235142827033997, -0.07755348831415176, 0.13637225329875946, -0.07296749949455261, 0.06672720611095428, -0.062297943979501724, 0.04247381538152695, -0.0979110524058342, 0.14036715030670166, 0.03178037703037262, 0.05660681053996086, 0.0879741981625557, 0.0668499618768692, -0.12904083728790283, 0.0935327559709549, 0.08089317381381989, -0.009003104642033577, 0.04552426561713219, -0.043035779148340225, 0.10019519925117493, -0.02000541239976883, 0.13623322546482086, 0.017425039783120155, -0.02419344335794449, 0.028840070590376854, -0.023922646418213844, 0.05940592288970947, 0.10041095316410065, -0.05530745908617973, -0.020832741633057594, -0.09561247378587723, 0.015455322340130806, 0.021557612344622612, 0.12421505898237228, 0.0703531801700592, 0.06005154922604561, -0.058770716190338135, 0.06287762522697449, -0.03740023821592331, 0.018962636590003967, -0.09380555897951126, -0.1182706281542778, -0.026249736547470093, 0.10059456527233124, 0.016445498913526535, -0.07592549175024033, 0.08783551305532455, 0.03997458145022392, -0.017483891919255257, 0.07993195205926895, -0.04334237053990364, -0.09069331735372543, 0.08725970983505249, -0.009572773240506649, 0.15104278922080994, 0.10680144280195236, -0.08494831621646881, 0.03168121352791786, 0.0510258674621582, 0.02936488203704357, 0.0957501232624054, -0.12437550723552704, -0.003967143129557371, 0.1070101335644722, -0.09869258850812912, 0.0752917006611824, -0.15034344792366028, 0.026343973353505135, -0.0928451344370842, -0.0567123182117939, -0.02041790448129177, 0.07985228300094604, 0.00257712765596807, -0.022665617987513542, 0.16601821780204773, -0.06240495666861534, -0.09321627765893936, -0.14882396161556244, -0.0383218452334404, -0.017351854592561722, 0.034503545612096786, -0.016470877453684807, 0.12539713084697723, -0.13373741507530212, 0.027005022391676903, 0.07497100532054901, -0.047416284680366516, 0.10646621137857437, 0.09922316670417786, -0.13090132176876068, -0.03835766017436981, 0.06281598657369614, -0.0015018410049378872, 0.14338846504688263, 0.10914045572280884, -0.09150093048810959, -0.13684730231761932, 0.11695052683353424, -0.10071782767772675, 0.026212353259325027, 0.08384812623262405, -0.05555097758769989, -0.11588161438703537, 0.04442804306745529, -0.07648028433322906, 0.08550599962472916, 0.09010384976863861, -0.075862355530262, 0.0470222607254982, 0.09249506145715714, 0.03337165340781212, -0.008087031543254852, -0.014172743074595928, -0.03590178117156029, -0.034017495810985565, 0.15645122528076172, -0.1349726915359497, -0.03167169168591499, 0.041960883885622025, 0.11653823405504227, -0.10617320239543915, -0.012593507766723633, 0.056428421288728714, -0.08567678183317184, -0.03399749472737312, 0.10398808121681213, 0.049252044409513474, -0.00257499935105443, 0.11627785116434097, -0.022616349160671234, -0.03272896632552147, 0.011658593080937862, 0.06642173230648041, 0.0860203430056572, -0.05391790345311165, -0.11807925254106522, 0.1385512351989746, -0.017482241615653038, 0.0355742946267128, -0.035882335156202316, -0.10860175639390945, 0.008706522174179554, 0.07058119028806686, 0.024719109758734703, -0.12082285434007645, 0.029215630143880844, 0.03832400217652321, 0.034713611006736755, -0.05175606533885002, 0.10090701282024384, 0.0968298390507698, 0.0762099176645279, -0.04719126597046852, -0.03017394430935383, -0.07803322374820709, 0.06431219726800919, -0.0981646478176117, -0.054596249014139175, 0.14042945206165314, 0.03603072464466095, 0.04968395084142685, -0.07316958159208298, 0.06798174232244492, -0.009645644575357437, 0.0912495031952858, -0.09688832610845566, -0.08040444552898407, 0.1300984025001526, 0.10059844702482224, -0.026219794526696205, -0.060419585555791855, 0.030846375972032547, 0.09691792726516724, 0.03636991232633591, 0.08725273609161377, -0.013512291945517063, 0.02662046253681183, -0.07894966006278992, -0.05822025611996651, 0.11416108906269073, 0.036472171545028687, 0.10377109795808792, -0.0955987200140953, 0.02401791885495186, -0.0506024993956089, -0.02798105590045452, -0.09412731975317001, -0.02692847140133381, -0.04769054427742958, 0.06768456101417542, -0.01730886474251747, -0.0718911662697792, -0.07917670160531998, 0.08380159735679626, 0.12840306758880615, -0.13420887291431427, 0.044350698590278625, 0.11167744547128677, -0.07666005194187164, 0.013566967099905014, 0.06129002571105957, 0.03078940510749817, -0.06787387281656265, 0.07733587175607681, 0.048444464802742004, 0.04905713349580765, -0.03713596612215042, 0.021410781890153885, 0.1107797622680664, -0.04101919010281563, -0.11333055049180984, 0.08464452624320984, -0.04832012951374054, -0.126077339053154, 0.07106941938400269, 0.09013890475034714, -0.06457976996898651, -0.0983680784702301, 0.0779004618525505, 0.08539550006389618, -0.026416253298521042, 0.055091965943574905, -0.09973467886447906, -0.034386046230793, -0.05136709660291672, -0.04401509091258049, 0.08979819715023041, 0.1399407833814621, -0.032774899154901505, 0.02776481769979, 0.057257600128650665, -0.05171504244208336, 0.037839844822883606, 0.09159375727176666, -0.08939693123102188, -0.07525990903377533, -0.03759419173002243, 0.00443398579955101, 0.10511736571788788, 0.037257175892591476, 0.0946103036403656, -0.11577607691287994, -0.06838960945606232, -0.08745138347148895, 0.08621588349342346, 0.028809182345867157, 0.02903776802122593, 0.07231201231479645, 0.07211408019065857, 0.09887036681175232, 0.1113753393292427, 0.017067749053239822, 0.05213828757405281, -0.01759037934243679, 0.035930078476667404, 0.11088692396879196, 0.05199280381202698, -0.10075124353170395, -0.10817098617553711, 0.06708696484565735, 0.009119313210248947, -0.053673673421144485, 0.02587929740548134, 0.00789080373942852, 0.016669631004333496, 0.05995162948966026, -0.04814052954316139, 0.11564800143241882, -0.08345384895801544, -0.0016077536856755614, 0.0672864019870758, 0.0844462439417839, 0.15869951248168945, -0.050021328032016754, 0.07139462977647781, -0.027559954673051834, -0.07685472071170807, 0.11939174681901932, -0.09171099215745926, -0.06386539340019226, 0.10691695660352707, 0.07776448130607605, 0.06658070534467697, -0.047016557306051254, 0.0787612572312355, 0.059504132717847824, 0.13031511008739471, 0.042178768664598465, -0.1054651290178299, 0.11071934551000595, 0.03174411877989769, 0.024047648534178734, -0.13369081914424896, 0.093605175614357, 0.0944925993680954, -0.008183915168046951, -0.026498762890696526, 0.03289780765771866, -0.017888767644762993, 0.08520247042179108, 0.04037255421280861, 0.12509895861148834, -0.08498882502317429, -0.10213270038366318, -0.04659548029303551, -0.01059652864933014, -0.12078242748975754, 0.0876164361834526, -0.053341854363679886, 0.011287804692983627, -0.1435518115758896, -0.040757257491350174, -0.0814962387084961, -0.11048143357038498, -0.08801241964101791, -0.05842774733901024, 0.10360094159841537, -0.008447443135082722, 0.11455971747636795, 0.028980238363146782, 0.010806089267134666, 0.059031084179878235, -0.07906000316143036, -0.020277705043554306, -0.021344130858778954, -0.14984124898910522, -0.013625207357108593, -0.10567273199558258, 0.14893054962158203, -0.06352008879184723, 0.11413227766752243, -0.0033845233265310526, 0.0004899177583865821, -0.074574775993824, 0.06548623740673065, 0.015440227463841438, 0.06186878681182861, 0.0826205238699913, -0.0973634198307991, 0.0808497741818428, -0.14046292006969452, -0.044161614030599594, -0.07096222043037415, -0.06996655464172363, 0.13842114806175232, -0.055912163108587265, 0.034037020057439804, -0.06770351529121399, 0.0398930124938488, -0.06914132833480835, 0.020145177841186523, 0.016276245936751366, 0.012918837368488312, -0.011557651683688164, 0.04619254171848297, -0.11255662143230438, -0.049311038106679916, -0.005687328986823559, 0.07394824922084808, 0.021184589713811874, 0.1193351075053215, -0.06598657369613647, -0.05304412916302681, -0.062363121658563614, 0.006949737202376127, 0.13130205869674683, -0.06364142894744873, 0.06638448685407639, -0.06308330595493317, -0.06074028089642525, 0.00994871649891138, 0.06255821138620377, 0.10702770203351974, 0.10860808938741684, -0.11902492493391037, -0.0012751729227602482, 0.04359104484319687, 0.08985254168510437, 0.11398400366306305, 0.08092740923166275, 0.0389031358063221, 0.028711697086691856, 0.13739177584648132, -0.06752603501081467, 0.08735347539186478, 0.028375951573252678, -0.08461128175258636, 0.057989366352558136, 0.02614760771393776, -0.04572759568691254, -0.06831664592027664, -0.050639934837818146, 0.15021216869354248, 0.01968356966972351, 0.1300085186958313, -0.06346888840198517, 0.0014778265031054616, -0.09449528902769089, -0.038255807012319565, 0.13739228248596191, 0.06289514154195786, -0.07186249643564224, -0.08732309937477112, -0.12938620150089264, -0.016856102272868156, -0.08952019363641739, 0.05028738081455231, 0.06714686751365662, -0.07672949135303497, 0.11166378855705261, -0.1348145455121994, -0.11329208314418793, 0.04601392522454262, 0.06938669085502625, -0.09118138253688812, -0.015903454273939133, -0.030653443187475204, -0.1123652532696724, 0.017458351328969002, 0.033169183880090714, 0.01884799264371395, 0.10938487946987152, -0.018046602606773376, 0.02275172248482704, 0.04133196920156479, 0.0345638133585453, -0.04313308745622635, -0.0834122970700264, -0.021205106750130653, -0.03364373743534088, -0.032996729016304016, -0.03419879823923111, 0.09739356487989426, -0.1018323227763176, 0.10699266940355301, -0.04661809280514717, 0.07046753168106079, -0.005914516746997833, -0.07967161387205124, 0.0698142871260643, 0.024594765156507492, -0.11400436609983444, 0.008158470503985882, -0.027794955298304558, -0.09433870017528534, 0.10434187203645706, 0.028491046279668808, 0.028706079348921776, -0.06791514903306961, 0.04066816344857216, -0.07187177240848541, -0.14469657838344574, -0.04973814636468887, 0.03613227605819702, -0.07037698477506638, 0.11246053874492645, 0.10550161451101303, -0.03893570974469185, -0.02940410189330578, 0.14613373577594757, 0.061486560851335526, -0.11202270537614822, 0.08919699490070343, -0.07197415828704834, -0.08072124421596527, -0.11386311054229736, -0.05922738090157509, 0.05148124694824219, 0.02292144112288952, -0.05002443119883537, 0.10114593803882599, 0.0998549535870552, -0.0775681883096695, 0.08777395635843277, 0.051369812339544296, 0.06035260111093521, 0.004304518923163414, 0.01995428279042244, -0.06211709976196289, 0.11450432240962982, -0.10504984855651855, 0.026298407465219498, 0.046772707253694534, -0.13386328518390656, -0.09612197428941727, 0.01967068575322628, 0.06863574683666229, 0.10745710879564285, 0.013064946979284286, -0.09543284773826599, -0.028173284605145454, -0.05374284088611603, 0.08557657152414322, 0.0026371830608695745, -0.014608488418161869, -0.056534234434366226, 0.037613801658153534, -0.03113899566233158, -0.04535859823226929, -0.07024203985929489, 0.08684639632701874, -0.027082398533821106, -0.030058080330491066, 0.11595285683870316, 0.14242522418498993, -0.09388723969459534, 0.12668785452842712, 0.09900388866662979, 0.13032899796962738, -0.1242561936378479, 0.022600416094064713, -0.03406127914786339, 0.06127452105283737, -0.12004322558641434, -0.07000035792589188, -0.057794611901044846, 0.06755475699901581, -0.026515303179621696, 0.04313094541430473, -0.09036841988563538, -0.07137112319469452, 0.026339225471019745, 0.051086269319057465, 0.0474066287279129, 0.08536004275083542, 0.07411899417638779, -0.0076173837296664715, 0.10639583319425583, -0.059137601405382156, -0.02093544788658619, -0.008973443880677223, 0.04991552233695984, -0.009642868302762508, 0.04417942836880684, 0.05338016524910927, -0.006227879785001278, 0.05138020217418671, 0.033012691885232925, 0.13084550201892853, 0.0038279457949101925, -0.00830602552741766, -0.11299271136522293, -0.09879197180271149, -0.10715901106595993, -0.02490503340959549, -0.06818556040525436, 0.10118419677019119, -0.01818057894706726, -0.006691721733659506, 0.11635041981935501, 0.03446691855788231, -0.08799570798873901, -0.010225252248346806, 0.05992937088012695, -0.0888729840517044, -0.06512436270713806, -0.047462355345487595, -0.058609601110219955, 0.03933209553360939, -0.006269871722906828, -0.13178755342960358, 0.008766277693212032, 0.09162136167287827, -0.10098179429769516, -0.09272732585668564, 0.06001787260174751, 0.08372481167316437, 0.12155728042125702, 0.017876742407679558, -0.019301559776067734, -0.01914113573729992, -0.09100537747144699, 0.023956475779414177, -0.04215046018362045, -0.07545847445726395, 0.1143803745508194, -0.06867101788520813, 0.025626907125115395, -0.007138197775930166, -0.04224099963903427, 0.10784672945737839, 0.05233273655176163, -0.07070253789424896, 0.10233908146619797, 0.059133417904376984, -0.024659041315317154, 0.12903448939323425, 0.15023936331272125, 0.03187253698706627, -0.018793106079101562, -0.1256163865327835, -0.080062635242939, -0.09899447858333588, -0.08898428082466125, -0.14634938538074493, 0.06894155591726303, 0.1110626608133316, 0.024865103885531425, -0.019589819014072418, 0.08410950005054474, -0.08026519417762756, 0.040438417345285416, 0.017711259424686432, 0.0042316121980547905, -0.017267778515815735, -0.07795989513397217, 0.10558606684207916, -0.08680372685194016, 0.06230437755584717, 0.014044001698493958, -0.0044097742065787315, -0.07752884179353714, -0.11942235380411148, 0.042366426438093185, -0.09809895604848862, 0.030283581465482712, 0.09259488433599472, 0.09719996154308319, -0.011966219171881676, -0.10378182679414749, 0.1041482463479042, 0.07393680512905121, 0.054408010095357895, 0.06370316445827484, 0.06670012325048447, -0.007476824335753918, -0.028708646073937416, 0.08271536231040955, 0.08687970787286758, 0.08528145402669907, -0.015371316112577915, -0.05232163518667221, 0.01281760260462761, 0.043974582105875015, -0.006582431495189667, -0.03744383901357651, -0.014048540964722633, 0.00423838198184967, 0.04430509731173515, -0.0989692360162735, 0.003404454793781042, -0.06900259107351303, 0.07748965919017792, -0.12488911300897598, 0.03253825753927231, -0.08173753321170807, -0.06814662367105484, -0.09515323489904404, 0.09673762321472168, -0.10162719339132309, -0.04844943806529045, -0.05730573832988739, 0.05216963589191437, -0.09998642653226852, -0.07818256318569183, -0.11053118854761124, 0.03388325124979019, 0.05405358970165253, -0.0849359855055809, 0.09061602503061295, 0.07722348719835281, -0.014066500589251518, 0.09541858732700348, -0.06348799914121628, -0.070215605199337, -0.009561761282384396, 0.07182639092206955, -0.0841163694858551, 0.00891652051359415, -0.0987551212310791, -0.01756007969379425, -0.08431944251060486, -0.04675515741109848, 0.03726634755730629, 0.024845030158758163, 0.04759994149208069, -0.06335268914699554, -0.010808147490024567, -0.06250929832458496, 0.07123538106679916, 0.0750991478562355, -0.09156598895788193, -0.12565644085407257, 0.07173280417919159, -0.05725252255797386, -0.034068141132593155, 0.09804752469062805, -0.01304344180971384, -0.04162340611219406, 0.07182271033525467, -0.035150956362485886, 0.11785990744829178, -0.03575234115123749, -0.02071777544915676, 0.0020531369373202324, 0.0692305937409401, 0.08893126994371414, 0.019998174160718918, -0.06200254708528519, -0.04798436909914017, 0.055586837232112885, 0.12040157616138458, 0.06551242619752884, -0.082173191010952, -0.023892322555184364, -0.06622742861509323, -0.04272693395614624, -0.016631178557872772, 0.02440875954926014, 0.025657186284661293, -0.05326462164521217, 0.059608157724142075, 0.011367719620466232, 0.07537250220775604, -0.04838988184928894, -0.012040945701301098, 0.0028239027597010136, -0.04811789467930794, 0.04444139823317528, 0.028601471334695816, 0.04051835462450981, 0.06521918624639511, -0.10015411674976349, 0.05760869011282921, 0.03618096560239792, -0.0466131791472435, -0.03511819988489151, -0.09276788681745529, -0.02183721773326397, -0.0972834900021553, -0.085383839905262, 0.014534743502736092, 0.006775469519197941, -0.03201160207390785, -0.08437585085630417, -0.0742468610405922, 0.002714066067710519, -0.06724284589290619, 0.04385988041758537, -0.07871878147125244, 0.008543367497622967, -0.11992082744836807, 0.0758204236626625, -0.07463754713535309, -0.10729946941137314, 0.0732528567314148, 0.0075567131862044334, -0.08537649363279343, -0.020439906045794487, 0.08140968531370163, -0.0938112735748291, 0.05660386383533478, 0.008336221799254417, -0.108602374792099, -0.029954666271805763, 0.08836282789707184, -0.08780784159898758, 0.09157092124223709, -0.04945182055234909, -0.013411165215075016, 0.10622622072696686, -0.06708959490060806, 0.024037078022956848, 0.00033835708745755255, 0.07721898704767227, -0.11940931528806686, 0.14170703291893005, -0.0704163908958435, -0.08494356274604797, 0.13490575551986694, -0.06988532096147537, 0.006190815474838018, 0.16543374955654144, 0.03590761125087738, -0.08073326200246811, -0.027846690267324448, 0.10128519684076309, -0.11266233772039413, -0.028623469173908234, 0.013646488077938557, 0.011659943498671055, 0.09808366745710373, -0.06875652074813843, -0.038753658533096313, 0.02057262510061264, 0.06261640042066574, -0.118462935090065, 0.11007809638977051, -0.03848490118980408, 0.025353847071528435, -0.11450780928134918, -0.07005635648965836, 0.06318720430135727, 0.09748022258281708, 0.12339182943105698, -0.02575000561773777, -0.04998775199055672, 0.061838697642087936, -0.040147390216588974, 0.1125912070274353, 0.013562719337642193, -0.045298755168914795, 0.02430209144949913, 0.05363850295543671, -0.11403992027044296, 0.14735594391822815, 0.16099753975868225, 0.011826368980109692, 0.0096066202968359, 0.05644411966204643, -0.07686850428581238, 0.06406420469284058, -0.017569903284311295, 0.04821710288524628, -0.012079811654984951, 0.07015149295330048, 0.0016312978696078062, 0.04146162420511246, -0.09010475873947144, -0.12750910222530365, 0.08102388679981232, -0.03927873075008392, 0.12521716952323914, -0.08567191660404205, 0.1282670646905899, -0.09197024255990982, -0.1285235732793808, 0.013987292535603046, -0.07327564805746078, 0.04439397156238556, 0.09048110246658325, -0.09069035202264786, -0.03131289407610893, 0.13352657854557037, -0.10867457091808319, -0.13449743390083313, -0.01483333483338356, -0.061543095856904984, -0.030384620651602745, -0.04572524502873421, 0.11047079414129257, 0.008435836061835289, 0.05416339263319969, -0.03896984085440636, 0.02801632322371006, -0.1008211076259613, 0.020933711901307106, -0.04193120077252388, 0.03208913281559944, 0.1268637478351593, 0.07671534270048141, 0.06446187198162079, -0.11032503098249435, -0.13375677168369293, -0.0522216372191906, -0.13611947000026703, -0.0757322758436203, -0.10535693913698196, 0.02023143880069256, -0.13693572580814362, -0.11622928082942963, 0.0953865721821785, 0.05174930766224861, -0.07099684327840805, -0.0283518023788929, 0.04014555737376213, -0.054114148020744324, 0.03373926877975464, -0.04844331741333008, -0.04118240997195244, -0.06839251518249512, -0.05424511060118675, -0.04703998193144798, -0.09870316833257675, -0.1394127905368805, 0.014690275304019451, -0.0001998583902604878, 0.08796927332878113, 0.08217082172632217, 0.06462040543556213, 0.07934187352657318, 0.014089786447584629, -0.060027267783880234, 0.07487732172012329, 0.12723051011562347, -0.09594554454088211, 0.08676199615001678, 0.058833107352256775, 0.03906514495611191, 0.024162381887435913, -0.07343370467424393, -0.014203757047653198, -0.14269743859767914, 0.0024970979429781437, 0.16083674132823944, 0.11064967513084412, 0.03847452998161316, -0.10675817728042603, -0.012752080336213112, 0.00852199736982584, 0.10470925271511078, -0.005772223696112633, -0.07626856118440628, 0.1083698496222496, -0.01030207984149456, 0.07242181152105331, -0.0008759304182603955, 0.05379682034254074, -0.13469018042087555, -0.08083163946866989, 0.08385822921991348, -0.11230086535215378, -0.04997442662715912, 0.10370306670665741, -0.017936566844582558, 0.022815721109509468, -0.05424265190958977, -0.12727446854114532, -0.06633251160383224, -0.057114377617836, 0.03959251940250397, -0.13223212957382202, 0.01267854031175375, 0.013864661566913128, -0.08644887804985046, -0.1133074164390564, 0.04807977005839348, -0.1206974908709526, 0.02914135903120041, 0.029202301055192947, 0.06204983592033386, 0.03364502266049385, -0.1101689487695694, -0.13668754696846008, -0.07318911701440811, -0.030584033578634262, -0.08909711241722107, -0.1303272247314453, 0.005696645937860012, 0.06961489468812943, 0.08906028419733047, -0.09996787458658218, 0.007947833277285099, -0.1274842768907547, 0.16678307950496674, -0.013136743567883968, 0.0512729212641716, 0.06685496121644974, -0.07921800762414932, -0.13307972252368927, -0.10939981788396835, -0.035574402660131454, -0.06509654968976974, 0.08890928328037262, 0.07385462522506714, 0.03884273022413254, -0.06387561559677124, 0.07437479496002197, 0.09736233204603195, -0.1076243594288826, 0.014557828195393085, -0.09137658029794693, 0.07107441872358322, -0.05200345814228058, -0.10029266029596329, 0.09131656587123871, -0.09713857620954514, 0.013949581421911716, 0.13860037922859192, -0.06439479440450668, 0.02541319653391838, 0.08156748861074448, -0.0783870592713356, -0.0483485609292984, 0.08668891340494156, -0.1127040907740593, 0.059324584901332855, -0.13578352332115173, 0.0015373085625469685, 0.07730380445718765, 0.08287443220615387, -0.13000550866127014, 0.03805379197001457, -0.06905823200941086, 0.06946631520986557, -0.09427158534526825, -0.04772770777344704, 0.12016452848911285, 0.10392549633979797, -0.05887095630168915, -0.10956021398305893, -0.11856617033481598, -0.07226242870092392, -0.10534481704235077, 0.07126810401678085, 0.03476359322667122, 0.00888559129089117, 0.057182300835847855, 0.014158605597913265, -0.09598572552204132, 0.036686137318611145, 0.0029326705262064934, 0.029393645003437996, 0.07479420304298401, -0.1026456207036972, -0.10144366323947906, -0.053926438093185425, -0.08109518885612488, 0.06966204196214676, -0.09774139523506165, -0.03993041068315506, 0.061722174286842346, -0.014793717302381992, -0.1072467491030693, -0.06594714522361755, -0.09239915758371353, 0.08813696354627609, -0.026483478024601936, 0.039234619587659836, -0.00540479552000761, 0.03152187541127205, -0.12841953337192535, -0.03883425518870354, 0.005981689319014549, -0.07207053154706955, 0.13225334882736206, -0.03353297710418701, 0.04890207201242447, 0.0547250360250473, -0.06202002987265587, 0.010398074053227901, -3.170210402458906e-05, -0.09907407313585281, -0.08985043317079544, 0.06149732321500778, 0.05312049388885498, -0.023525040596723557, -0.031925708055496216, 0.0521949864923954, -0.003989395685493946, 0.07162373512983322, -0.04099937155842781, 0.02657606080174446, 0.051414910703897476, -0.10246288776397705, 0.01504516415297985, -0.11083712428808212, 0.023523572832345963, -0.10681553184986115, 0.04055815935134888, 0.04769842326641083, 0.06417558342218399, -0.013114500790834427, -0.0260368250310421, 0.051626525819301605, -0.0059013632126152515, -0.004512842278927565, 0.07467184960842133, -0.0554841123521328, -0.005286803934723139, 0.0014193763490766287, -0.04082440212368965, 0.10556692630052567, 0.002965982537716627, 0.03612981364130974, -0.10535016655921936, -0.011577594093978405, -0.02468898519873619, -0.09611485153436661, 0.07689747214317322, 0.0743669867515564, -0.02624543011188507, 0.06205378845334053, -0.023616749793291092, 0.01569424383342266, -0.1138402447104454, -0.02140585333108902, -0.04065195843577385, -0.06648468971252441, 0.09863279014825821, 0.12573713064193726, 0.05168229714035988, 0.02937200292944908, 0.07791324704885483, -0.08000113815069199, -0.020108170807361603, 0.08438825607299805, 0.005832782480865717, -0.12107101082801819, -0.04122067987918854, 0.05215638875961304, -0.022287050262093544, -0.053629182279109955, -0.13336917757987976, 0.09354265034198761, -0.017538338899612427, 0.06727509200572968, 0.042995210736989975, 0.033071015030145645, 0.013385401107370853, 0.04489072784781456, -0.04058206081390381, -0.07374267280101776, 0.021675609052181244, -0.05308447778224945, 0.013357418589293957, -0.03508362919092178, -0.11962452530860901, 0.031291279941797256, -0.056492023169994354, 0.06471256911754608, -0.044747792184352875, 0.12492956221103668, 0.07679461687803268, -0.0022128999698907137, 0.030583247542381287, 0.12797458469867706, 0.061547018587589264, 0.0934031531214714, -0.06053325533866882, -0.11527115851640701, -0.038726553320884705, 0.04056762158870697, 0.09105852246284485, -0.13197605311870575, 0.04277165234088898, 0.10895856469869614, -0.14695551991462708, -0.08359241485595703, -0.08591014891862869, -0.12868161499500275, -0.018194440752267838, -0.071194127202034, 0.08841543644666672, -0.0020248829387128353, -0.0764792338013649, 0.12712571024894714, -0.07325625419616699, -0.03279178589582443, 0.09350401163101196, -0.09564928710460663, -0.0780211091041565, 0.08214356750249863, -0.005417789798229933, -0.13295139372348785, -0.13132883608341217, -0.10594610124826431, 0.02986157312989235, 0.027401305735111237, -0.03907974436879158, 0.07613606750965118, 0.14831428229808807, 0.0424489788711071, -0.01786159723997116, -0.03535782918334007, -0.028047196567058563, -0.024875614792108536, 0.07417907565832138, -0.004678857512772083, -0.038702141493558884, -0.018147066235542297, 0.0786421149969101, 0.06306518614292145, -0.027509862557053566, -0.09090393781661987, 0.04689320921897888, 0.08588725328445435, -0.05480511486530304, -0.0424591489136219, -0.03398606553673744, 0.010389992035925388, 0.10669481754302979, 0.10687645524740219, 0.010703061707317829, 0.055397287011146545, -0.03431940823793411, 0.04596131294965744, -0.10502630472183228, -0.07457128167152405, -0.031426023691892624, 0.05759508162736893, -0.03220401704311371, -0.029390772804617882, 0.12537768483161926, 0.06832549721002579, 0.032425977289676666, 0.0608159676194191, 0.07853556424379349, 0.01610914245247841, -0.06772579997777939, -0.007259904872626066, -0.07471102476119995, 0.09977923333644867, 0.0030158960726112127, -0.08313893526792526, -0.0361771397292614, 0.04107974097132683, 0.027651283890008926, -0.08968012034893036, -0.012135694734752178, 0.03622660040855408, -0.09361699223518372, 0.024058837443590164, -0.033459246158599854, 0.02299254946410656, -0.056058209389448166, 0.06294135749340057, 0.08261996507644653, 0.04903717339038849, -0.11280912160873413, 0.034943852573633194, 0.045481640845537186, 0.030025802552700043, -0.06132031977176666, 0.057840634137392044, -0.11641988158226013, 0.008792274631559849, 0.053335726261138916, 0.06387612223625183, -0.04406982660293579, -0.05670303478837013, 0.11338771134614944, 0.06925235688686371, -0.016872966662049294, 2.281030174344778e-05, -0.015667885541915894, 0.05146077647805214, -0.01799919828772545, -0.06275703012943268, 0.012651899829506874, -0.04083649069070816, -0.03533778712153435, -0.09948141872882843, 0.06374677270650864, 0.04602343216538429, 0.0276592206209898, 0.09162436425685883, 0.11604346334934235, 0.03218501806259155, 0.03225881978869438, -0.07542544603347778, 0.03749649226665497, 0.09141344577074051, 0.03627437725663185, 0.0754946619272232, -0.0034550761338323355, -0.14341987669467926, -0.05927980691194534, -0.08200439810752869, 0.015090467408299446, -0.09016617387533188, -0.04299343749880791, 0.10836336016654968, -0.0367165170609951, -0.0540630966424942, -0.08389422297477722, -0.056205425411462784, -0.10532315820455551, -0.06224793568253517, -0.018347805365920067, -0.09168041497468948, 0.059069253504276276, -0.01914135552942753, -0.04841075465083122, 0.11662587523460388, -0.09436842799186707, 0.04209202155470848, -0.019763341173529625, 0.09695322066545486, -0.04934521019458771, 0.04702925682067871, -0.07535456866025925, 0.10458924621343613, -0.090556800365448, 0.05244026333093643, 0.015344202518463135, 0.04272376745939255, 0.07425481826066971, 0.01963183656334877, -0.006788038648664951, 0.05855116993188858, 0.12353449314832687, 0.04000014066696167, 0.07529979199171066, 0.1160421296954155, -0.09663943201303482, -0.12179096788167953, 0.011649281717836857, -0.01740998588502407, 0.0014376463368535042, 0.08449865877628326, -0.09057489782571793, -0.11501059681177139, -0.07738092541694641, -0.10593193769454956, 0.1170380711555481, 0.05427880212664604, -0.07275918126106262, 0.01845335215330124, 0.01949952356517315, -0.04982667788863182, -0.007161690853536129, -0.017411556094884872, -0.06681939959526062, 0.016040170565247536, 0.08600340783596039, -0.03878084570169449, -0.05904075503349304, 0.015623440966010094, 0.0028787481132894754, 0.1069154143333435, 0.057211827486753464, 0.044148221611976624, 0.11583540588617325, 0.05193508788943291, -0.08418663591146469, -0.0768648087978363, -0.10315530747175217, 0.012116552330553532, 0.0032839705236256123, -0.08936629444360733, -0.13170452415943146, 0.10398194938898087, 0.09007123857736588, -0.08540089428424835, -0.022366708144545555, -0.07312167435884476, -0.018334263935685158, 0.04006000608205795, -0.07919944077730179, -0.11934906989336014, -0.061574868857860565, -0.049972452223300934, 0.05963142216205597, 0.05013909190893173, -0.06268878281116486, 0.02628319337964058, 0.059342432767152786, -0.03766437619924545, -0.0979577898979187, 0.06031714379787445, -0.11578674614429474, -0.025683993473649025, -0.01474730484187603, -0.0036740959621965885, 0.09385369718074799, 0.006767645012587309, 0.03235455974936485, 0.006998011376708746, 0.027748290449380875, -0.13229398429393768, -0.13110177218914032, 0.02121940813958645, 0.022559115663170815, 0.07154172658920288, 0.1272200644016266, -0.12736786901950836, 0.009192429482936859, 0.108230821788311, 0.09351327270269394, 0.09074664860963821, -0.07092083990573883, -0.0014425136614590883, 0.04763488844037056, 0.03907182440161705, 0.041988715529441833, 0.04707752540707588, 0.010601961985230446, -0.09139668941497803, 0.10216139256954193, -0.031209221109747887, 0.021672194823622704, -0.06416457146406174, -0.05391702428460121, -0.04068291187286377, 0.06665299832820892, -0.055946920067071915, 0.0038660268764942884, 0.012289445847272873, -0.043294209986925125, -0.05992971360683441, -0.026517005637288094, -0.0007326401537284255, -0.04339168220758438, -0.03661217913031578, 0.028129827231168747, -0.07077690213918686, 0.07842113077640533, -0.03865040838718414, -0.025676827877759933, -0.08801013976335526, 0.04458038508892059, -0.018224431201815605, 0.054489973932504654, -0.09838882088661194, 0.054351966828107834, -0.02487233839929104, -0.033898670226335526, -0.10471726953983307, 0.0033301508519798517, -0.12043772637844086, -0.05357653275132179, -0.018161972984671593, -0.007337925955653191, -0.03332802280783653, 0.10502231866121292, -0.005124549847096205, -0.05275173857808113, -0.005408216267824173, -0.0861404612660408, -0.05143085494637489, -0.012678288854658604, -0.07019240409135818, -0.09576054662466049, -0.08086206763982773, 0.018906334415078163, -0.02240571938455105, -0.00437762588262558, -0.021490300074219704, 0.10320223867893219, -0.0006997595191933215, 0.1075262799859047, 0.07109106332063675, 0.020633114501833916, 0.10246291011571884, 0.11385199427604675, -0.040319688618183136, -0.03698557987809181, -0.05831395462155342, -0.08434561640024185, 0.02417476661503315, 0.0015287711285054684, -0.0351334884762764, -0.11749891936779022, -0.048790398985147476, -0.06721388548612595, 0.08061806112527847, -0.05853968858718872, 0.0009537829319015145, -0.04788225144147873, 0.14324064552783966, -0.044834237545728683, -0.11384481191635132, 0.016623446717858315, 0.09754014760255814, -0.02783321961760521, 0.09837117791175842, 0.031377505511045456, -0.052979208528995514, -0.10426917672157288, 0.10982174426317215, 0.006657541263848543, -0.02356811799108982, -0.030553214251995087, 0.1388051062822342, 0.12782447040081024, 0.08409470319747925, -0.11354795098304749, 0.08143067359924316, 0.0981360450387001, 0.0667867586016655, 0.0936703234910965, -0.03325361758470535, 0.05098458752036095, -0.11828049272298813, 0.005965002346783876, -0.08940065652132034, 0.09474401921033859, -0.056163206696510315, -0.11937186866998672, 0.004615938290953636, -0.056602779775857925, -0.08568327873945236, -0.051415570080280304, -0.03214295208454132, 0.0869455337524414, -0.10060286521911621, 0.06350520253181458, -0.10481049120426178, 0.0057249050587415695, 0.0025528075639158487, 0.004797198344022036, -0.010575877502560616, -0.003521039616316557, 0.050778377801179886, 0.006907357834279537, 0.05244140326976776, 0.025315284729003906, -0.02076817862689495, -0.06769398599863052, -0.008122731000185013, -0.01855526864528656, 0.017854347825050354, 0.041678667068481445, -0.14119726419448853, 0.09971597790718079, 0.05545628443360329, -0.08378405123949051, 0.08680172264575958, -0.011857228353619576, -0.059720829129219055, -0.05039798840880394, -0.04182465001940727, 0.10009446740150452, 0.07446672022342682, 0.08302009850740433, -0.046797361224889755, 0.09194397181272507, -0.00939746480435133, -0.046216756105422974, -0.006661254912614822, 0.08606521785259247, 0.08513356745243073, -0.01838103123009205, 0.08380081504583359, -0.022483205422759056, 0.032366856932640076, -0.024060875177383423, -0.11206459254026413, 0.06434187293052673, -0.0883449837565422, -0.07055124640464783, -0.06024807691574097, -0.06669069081544876, -0.009616991505026817, -0.020561572164297104, 0.024390388280153275, 0.0020378699991852045, 0.13945917785167694, 0.11214645206928253, 0.024392761290073395, -0.0006774655776098371, -0.014830067753791809, -0.04257811978459358, 0.09388148039579391, -0.05317942798137665, -0.11521565169095993, 0.0851932018995285, 0.0038953686598688364, 0.05710462108254433, 0.13096743822097778, -0.09021048247814178, -0.066850945353508, 0.01584049127995968, -0.10070819407701492, 0.08909600228071213, 0.0641566663980484, 0.07571262866258621, -0.06932471692562103, -0.09435392916202545, -0.03042048029601574, -0.01871493086218834, 0.08867359161376953, -0.08333279937505722, -0.02333659492433071, -0.10512956976890564, -0.030604010447859764, 0.06834488362073898, 0.0924651101231575, -0.06659507006406784, 0.09011706709861755, 0.04511943832039833, 0.12895716726779938, -0.0791277214884758, 0.04888670891523361, 0.0008856207714416087, -0.0577767938375473, -0.02862047590315342, 0.02220352739095688, -0.057534441351890564, -0.0760660246014595, -0.023975947871804237, -0.029422253370285034, -0.036586321890354156, -0.01095549762248993, -0.10514884442090988, -0.06781649589538574, -0.0890890583395958, -0.0320155993103981, 0.13725557923316956, 0.010393692180514336, -0.027530893683433533, -0.0064967116340994835, 0.023351594805717468, -0.014085369184613228, 0.06697383522987366, -0.06351788341999054, -0.015681138262152672, 0.10323337465524673, -0.09144420921802521, 0.10698415338993073, -0.1101117879152298, -0.07937546819448471, 0.11811826378107071, 0.011414889246225357, 0.09515567868947983, 0.09158395975828171, -0.03342750295996666, -0.009918038733303547, 0.0316278412938118, 0.11188847571611404, 0.09290407598018646, -0.08264680206775665, 0.038097526878118515, 0.07976875454187393, 0.08141427487134933, -0.06730091571807861, 0.04884422570466995, 0.08070971071720123, 0.040580328553915024, -0.12284893542528152, 0.05456164851784706, 0.10035311430692673, 0.03173277899622917, -0.02655480057001114, 0.014612670987844467, 0.009798571467399597, 0.04816802218556404, 0.11066468060016632, -0.06079728901386261, -0.0011009437730535865, -0.10057727247476578, -0.05584723502397537, -0.013171478174626827, -0.01755959913134575, 0.08980517089366913, -0.0171152763068676, -0.06403938680887222, -0.033775266259908676, -0.08763717859983444, 0.0028058092575520277, -0.07684510946273804, 0.0007494754972867668, -0.08371856063604355, 0.004817646462470293, 0.010085389949381351, 0.04335581138730049, -0.08480509370565414, -0.10885411500930786, -0.0007737716659903526, 0.04179656505584717, 0.04155021533370018, -0.08948152512311935, -0.021843627095222473, -0.1066385880112648, -0.11764374375343323, 0.07666204124689102, -0.03505334630608559, 0.019106680527329445, -0.032183170318603516, 0.005369722843170166, 0.0746859535574913, 0.1187627911567688, -0.0652102380990982, 0.14120176434516907, 0.0017907643923535943, -0.06459230184555054, 0.01430326234549284, 0.03307504951953888, 0.0004908787668682635, 0.0432218499481678, -0.028234703466296196, -0.07842791825532913, 0.15213197469711304, -0.09127278625965118, -0.036090683192014694, -0.07251749187707901, -0.07635761052370071, 0.06419511139392853, -0.006275725085288286, -0.1222391426563263, -0.06034264340996742, 0.014435242861509323, -0.09061455726623535, -0.053302064538002014, -0.021046169102191925, 0.0939161404967308, 0.009172382764518261, 0.1191011294722557, -0.10154883563518524, 0.027630848810076714, -0.007733736652880907, 0.02823844738304615, 0.07231047749519348, 0.014985564164817333, -0.07849586755037308, 0.0626121535897255, -0.009688550606369972, 0.03873611241579056, -0.09050824493169785, -0.056042734533548355, -0.00011023097613360733, 0.0036941256839782, -0.021626098081469536, -0.118467777967453, -0.09300575405359268, 0.031155509874224663, -0.03771740198135376, -0.10065428912639618, 0.030234653502702713, 0.06907134503126144, 0.06703260540962219, -0.046798381954431534, -0.051039014011621475, -0.03800734877586365, 0.10474811494350433, 0.01465584896504879, -0.05114451050758362, -0.060664720833301544, 0.06773149222135544, 0.05464434251189232, 0.003602428827434778, -0.012730342335999012, -0.043403469026088715, 0.06878485530614853, 0.09862608462572098, 0.06403310596942902, -0.045381948351860046, -0.029714006930589676, 0.018277660012245178, 0.05323522165417671, 0.052630383521318436, 0.08515744656324387, -0.009008729830384254, -0.00754893571138382, -0.0751146674156189, -0.02163948491215706, -0.12025375664234161, -0.011645885184407234, -0.09926553815603256, 0.1430133730173111, -0.08958174288272858, 0.11392391473054886, 0.04382006824016571, 0.08126440644264221, 0.04182901233434677, -0.007555618416517973, -0.02342280000448227, -0.03456372395157814, -0.07377900928258896, -0.058066677302122116, -0.025884749367833138, -0.05274621769785881, 0.06652452051639557, 0.014168049208819866, 0.08951100707054138, 0.029266640543937683, -0.03391106799244881, 0.020975880324840546, -0.10177601128816605, -0.0611582025885582, 0.07477111369371414, 0.10467644780874252, 0.12736856937408447, 0.011046863161027431, -0.012368564493954182, -0.04737616702914238, 0.017909983173012733, -0.02221018075942993, 0.13891275227069855, 0.1439974457025528, 0.12421276420354843, -0.08911856263875961, 0.10564520955085754, 0.06712423264980316, -0.011813035234808922, 0.05425193905830383, -0.08958268910646439, 0.02659677527844906, -0.017535563558340073, 0.01679358072578907, 0.080693818628788, 0.09976442903280258, 0.0017178063280880451, 0.02209823951125145, 0.043331459164619446, 0.014735436998307705, 0.07401679456233978, -0.023919129744172096, -0.04271918907761574, -0.027236925438046455, -0.07966071367263794, -0.08437540382146835, -0.10557626187801361, -0.07977467030286789, -0.09272485971450806, -0.07083281129598618, -0.05053937807679176, -0.06831187754869461, -0.03209080547094345, 0.04012083262205124, -0.08179298043251038, 0.10301008075475693, 0.021165091544389725, -0.010366037487983704, 0.053689491003751755, 0.1329805999994278, -0.06192680075764656, -0.005853992886841297, -0.09461703151464462, -0.04795251414179802, -0.01466920506209135, -0.11371166259050369, -0.0540095716714859, -0.07068116962909698, -0.09362789243459702, 0.11804773658514023, -0.04383338987827301, -0.03014596737921238, 0.13680961728096008, -0.10480804741382599, 0.05489065498113632, 0.013502880930900574, 0.05836695060133934, -0.07734890282154083, -0.002275224309414625, 0.09895651787519455, 0.0892261415719986, 0.04704881086945534, -0.05960782989859581, -0.10534097254276276, -0.06573078781366348, 0.012323345057666302, 0.040122102946043015, 0.02832583151757717, -0.010358648374676704, -0.07690468430519104, -0.09569699317216873, 0.0007191192707978189, 0.020162425935268402, 0.11851440370082855, 0.04411400109529495, -0.03346171975135803, 0.013603419065475464, 0.07717288285493851, 0.0033191549591720104, -0.01511898823082447, 0.07445862889289856, -0.10769297927618027, -0.07187897711992264, -0.027650240808725357, -0.027691202238202095, -0.1085595190525055, 0.01905151829123497, 0.025263594463467598, 0.10992171615362167, -0.0775703638792038, 0.12166553735733032, 0.13632075488567352, -0.03621672838926315, -0.049380164593458176, -0.013272499665617943, -0.018335646018385887, -0.07395382970571518, 0.1430869996547699, 0.09618697315454483, 0.09255045652389526, 0.06274411827325821, -0.07594692707061768, -0.003583087120205164, 0.033720917999744415, -0.04548017308115959, 0.13038888573646545, -0.005387831479310989, -0.09768784046173096, -0.04440510272979736, 0.09475795924663544, 0.03279249742627144, 0.07473496347665787, 0.1042948067188263, 0.07095711678266525, 0.028292223811149597, 0.010226459242403507, -0.003907449077814817, 0.07100902497768402, -0.10496242344379425, 0.007486374117434025, 0.11966029554605484, -0.09586527943611145, -0.004384729545563459, 0.1639130711555481, -0.028195325285196304, 0.04933694750070572, -0.11699756979942322, -0.10056867450475693, 0.01311455201357603, 0.06996692717075348, -0.015184590592980385, -0.013873465359210968, 0.06368999183177948, 0.044808339327573776, -0.02555757761001587, 0.11006137728691101, 0.04173159971833229, -0.10346581041812897, -0.12695957720279694, -0.09211600571870804, 0.13843202590942383, -0.021383237093687057, -0.08418194204568863, 0.06603250652551651, -0.14390972256660461, -0.045689910650253296, -0.009263839572668076, -0.01986292563378811, 0.04370592162013054, 0.08606259524822235, 0.022624710574746132, -0.07655210793018341, -0.06920719146728516, -0.07279720157384872, -0.10725880414247513, -0.01712646149098873, -0.07319549471139908, 0.09613131731748581, -0.06212545931339264, -0.12584218382835388, 0.04192661494016647, 0.10402669757604599, 0.0808202475309372, 0.027709251269698143, 0.06795024871826172, 0.008513227105140686, 0.0006928987568244338, -0.03021620213985443, -0.014856434427201748, 0.07405463606119156, -0.10843418538570404, -0.0630679577589035, 0.006724407430738211, -0.018684806302189827, -0.003963221795856953, 0.059074025601148605, 0.06592928618192673, 0.07089009135961533, 0.014674314297735691, 0.01877428963780403, 0.06226968392729759, -0.009605119936168194, 0.11213654279708862, 0.01007664855569601, -0.0025168920401483774, -0.0580170638859272, 0.05420803651213646, -0.019145362079143524, 0.01959458366036415, -0.12489662319421768, -0.008716331794857979, 0.04368065670132637, -0.04087277129292488, -0.12897244095802307, -0.12225250899791718, 0.039188314229249954, -0.01613282784819603, 0.1021285131573677, 0.017989762127399445, 0.1277427077293396, 0.04711295664310455, -0.10926681756973267, 0.026799503713846207, 0.0316455252468586, -0.029856672510504723, -0.10323290526866913, 0.027583038434386253, -0.06218550354242325, 0.000572307500988245, -0.06694460660219193, -0.09940191358327866, 0.012508946470916271, -0.09698288142681122, -0.09435509890317917, 0.10174618661403656, 0.07404204457998276, -0.022781625390052795, 0.039010729640722275, 0.0008682123152539134, 0.08247758448123932, -0.06016284599900246, 0.11442040652036667, 0.10159992426633835, 0.06311079114675522, 0.08322955667972565, -0.13524997234344482, -0.11936421692371368, 0.12966430187225342, 0.06689072400331497, -0.05711275711655617, -0.05396497622132301, -0.14390124380588531, 0.005953746382147074, 0.02091953344643116, 0.07277394831180573, -0.05579164996743202, -0.010880953632295132, 0.13633139431476593, -0.025126226246356964, -0.06353753060102463, 0.07914040237665176, 0.01767631620168686, 0.08033192902803421, 0.0945475772023201, -0.06224517524242401, -0.04895501583814621, -0.0506371334195137, -0.0259274709969759, 0.09034783393144608, 0.009797598235309124, -0.09448950737714767, -0.07603363692760468, -0.045423224568367004, 0.09321671724319458, 0.13098691403865814, -0.09601795673370361, 0.08410676568746567, 0.07877195626497269, -0.05141211301088333, -0.005311610642820597, 0.05428396537899971, -0.1256389319896698, 0.06457550823688507, 0.12790648639202118, 0.05297991260886192, 0.09670858830213547, 0.05505640059709549, -0.0803907960653305, 0.03296976163983345, -0.030248790979385376, 0.03773454576730728, -0.14358074963092804, 0.029157070443034172, -0.060937441885471344, 0.0394664965569973, -0.025185072794556618, 0.05686315894126892, -0.10355327278375626, -0.1344854086637497, 0.03110649064183235, -0.08058973401784897, 0.10854269564151764, 0.0761011391878128, 0.0029304728377610445, 0.06067744269967079, 0.010201963596045971, 0.023751990869641304, -0.10480093955993652, -0.0315810926258564, -0.07868821173906326, 0.06581675261259079, 0.08260002732276917, -0.07747591286897659, -0.015045871026813984, -0.07655851542949677, -0.06670516729354858, 0.05891025811433792, -0.02939406968653202, 1.081760274246335e-05, -0.07459325343370438, 0.03970377519726753, -0.10000170767307281, 0.009555497206747532, 0.06649728119373322, 0.09250465035438538, 0.1352388560771942, -0.05164887756109238, 0.02271316759288311, -0.04180814325809479, 0.08986169844865799, -0.07989925146102905, 0.036271832883358, -0.008915266022086143, 0.029742522165179253, 0.035768818110227585, -0.04880015552043915, 0.14105579257011414, 0.06405264139175415, -0.0485488697886467, -0.07780061662197113, 0.00022007452207617462, 0.011507619172334671, -0.13172496855258942, -0.06623673439025879, 0.005011535715311766, 0.10293024033308029, -0.06731900572776794, 0.030147088691592216, -0.005401295144110918, 0.06252805888652802, 0.02467125840485096, 0.006345808506011963, -0.010918832384049892, -0.08725382387638092, 0.12800049781799316, -0.021972831338644028, 0.019412817433476448, 0.0862971618771553, -0.14452911913394928, 0.011022078804671764, -0.04534241557121277, -0.15256267786026, 0.08171892166137695, -0.00865965150296688, 0.12443017214536667, -0.039721764624118805, -0.10036325454711914, 0.044212862849235535, -0.07580972462892532, -0.00044697028351947665, -0.030566509813070297, 0.02160581760108471, 0.07725916057825089, 0.09786535054445267, -0.021914251148700714, 0.019003206863999367, 0.05478506162762642, 0.09868614375591278, 0.03415770083665848, 0.012474407441914082, -0.1126914694905281, 0.04990757256746292, 0.04282033070921898, -0.08919574320316315, -0.000742304022423923, -0.10441483557224274, -0.006973642390221357, -0.09288335591554642, 0.09069359302520752, 0.08227739483118057, 0.06533949077129364, -0.06876622140407562, 0.08364350348711014, -0.01557222567498684, -0.11760858446359634, -0.10208485275506973, -0.02147124707698822, -0.03219063952565193, -0.12868082523345947, 0.08291155099868774, 0.01591535098850727, -0.017885688692331314, 0.04334401339292526, -0.041036155074834824, 0.049759391695261, 0.09352333843708038, 0.011481682769954205, 0.04975711181759834, 0.07312053442001343, -0.05451396852731705, -0.10499286651611328, -0.06843127310276031, 0.09446441382169724, 0.12349008023738861, -0.08784539997577667, 0.014873885549604893, -0.047758519649505615, 0.08371054381132126, 0.10485407710075378, -0.044329624623060226, 0.03654729574918747, 0.08895113319158554, -0.0007309181382879615, 0.0006268706638365984, -0.11689396947622299, 0.0418863408267498, -0.04346485808491707, 0.007340237032622099, 0.12530550360679626, 0.09754794090986252, 0.11654303967952728, -0.07502797991037369, -0.0861683115363121, 0.06748399883508682, -0.010995070450007915, -0.00710400240495801, -0.02060992270708084, -0.05018889158964157, -0.0304181519895792, -0.05117442086338997, -0.09939085692167282, -0.031257037073373795, 0.047804996371269226, 0.13760273158550262, -0.06358205527067184, -0.056408680975437164, -0.0843677967786789, -0.018429283052682877, -0.13201993703842163, -0.05779445171356201, 0.11618267744779587, -0.06389026343822479, 0.13630841672420502, 0.06044335663318634, -0.06101362407207489, 0.04216230660676956, -0.0961446613073349, 0.09602536261081696, 0.048945311456918716, -0.046367790549993515, -0.04111520200967789, -0.021309051662683487, -0.03228224813938141, -0.049614064395427704, -0.09202706068754196, -0.047754742205142975, -0.07287464290857315, 0.09085960686206818, 0.08771780133247375, 0.003930889070034027, -0.05716255307197571, -0.0071671949699521065, 0.11847783625125885, 0.017116157338023186, 0.09732095897197723, 0.11277252435684204, 0.10361640900373459, -0.08673839271068573, 0.04219662398099899, -0.020463569089770317, -0.0949842780828476, 0.07855349034070969, -0.10447975993156433, 0.09375649690628052, 0.0020736106671392918, 0.008447209373116493, 0.09220890700817108, 0.05866546556353569, 0.1123337596654892, 0.06968954205513, 0.11494257301092148, 0.0219129528850317, 0.13709400594234467, -0.04814419522881508, 0.035485200583934784, -0.13161590695381165, -0.03255350887775421, -0.006666943430900574, 0.06808515638113022, 0.08705711364746094, -0.05802634358406067, -0.017265304923057556, 0.03851291909813881, 0.08888544142246246, -0.09284097701311111, -0.017914090305566788, -0.07115627825260162, -0.051384638994932175, 0.02736087329685688, -0.04857755824923515, -0.029115840792655945, 0.019264405593276024, -0.025213664397597313, 0.07243257015943527, 0.06115811690688133, 0.010513867251574993, -0.06402792036533356, -0.015921689569950104, 0.021419299766421318, -0.03264565020799637, -0.08590074628591537, 0.01650119572877884, -0.07295599579811096, -0.04701842740178108, 0.04135008528828621, 0.022836145013570786, 0.010381309315562248, -0.10868586599826813, -0.0676208883523941, 0.058648452162742615, 0.03610135614871979, 0.10505443811416626, 0.025429077446460724, -0.03781847283244133, -0.10192907601594925, -0.0754779726266861, -0.10735765844583511, -0.016579071059823036, -0.08883991092443466, 0.041933923959732056, -0.11226622760295868, 0.12350435554981232, -0.03779572620987892, 0.03961788862943649, -0.020758699625730515, -0.000867804978042841, 0.03393565118312836, 0.0516514889895916, -0.10263056308031082, 0.022286690771579742, -0.08847538381814957, 0.10848933458328247, -0.054757729172706604, 0.07755313068628311, -0.06540422886610031, 0.03841809555888176, -0.08595311641693115, -0.056047216057777405, -0.05226483568549156, -0.028749577701091766, 0.06706923991441727, 0.009814330376684666, -0.05714527890086174, 0.0642060935497284, -0.08008123189210892, -0.09496981650590897, 0.029538635164499283, -0.010201244615018368, 0.1354503035545349, 0.06975757330656052, -0.14237545430660248, -0.05549179017543793, 0.08174439519643784, 0.05518646165728569, 0.0011606508633121848, -0.16174468398094177, 0.06203974410891533, -0.11479218304157257, -0.0023750171530991793, 0.07213356345891953, 0.032005880028009415, 0.023514265194535255, 0.15513485670089722, -0.047198791056871414, -0.033003464341163635, 0.07448410987854004, -0.01269129291176796, 0.002075783209875226, -0.03617887571454048, -0.021764228120446205, 0.07809382677078247, 0.11113077402114868, 0.05607610195875168, -0.11991243064403534, 0.07408798485994339, 0.07891183346509933, 0.001701624714769423, -0.09447254240512848, 0.067131906747818, -0.14172646403312683, -0.04795188084244728, 0.03758925199508667, 0.051131680607795715, 0.07074514776468277, -0.013260007835924625, -0.08892204612493515, -0.03289257735013962, 0.04850155860185623, -0.15018589794635773, -0.038115911185741425, -0.009889109060168266, 0.09295296669006348, 0.09731114655733109, 0.02084576152265072, -0.0447651632130146, 0.04487035050988197, 0.03805837780237198, 0.11767928302288055, 0.0743640884757042, 0.01916062831878662, -0.02502991072833538, -0.035023629665374756, 0.10989734530448914, -0.1104409471154213, 0.0594225637614727, 0.03717077523469925, 0.032807357609272, 0.059870149940252304, 0.09756317734718323, 0.04572640731930733, -0.09510644525289536, -0.0885152593255043, -0.04220915213227272, 0.08128809928894043, -0.04620761796832085, 0.028918562456965446, 0.039779044687747955, 0.010107732377946377, 0.03323781490325928, 0.0452953539788723, 0.07687141001224518, 0.040987126529216766, 0.07494650036096573, 0.04154084250330925, 0.09544452279806137, 0.026893338188529015, -0.07164528220891953, 0.10754804313182831, -0.13055981695652008, 0.09837073087692261, 0.021849364042282104, -0.1028151735663414, 0.07465510070323944, -0.10207609087228775, 0.04896077513694763, 0.007954980246722698, -0.05119748041033745, -0.062105342745780945, -0.10279329121112823, -0.031758591532707214, 0.007648795377463102, 0.1498790681362152, -0.07445137202739716, -0.06172868609428406, -0.07657555490732193, -0.06666118651628494, 0.007121214177459478, -0.04650377854704857, -0.05811233073472977, -0.04135958477854729, 0.08042304217815399, -0.08645496517419815, 0.057605769485235214, 0.0824350044131279, 0.03294341266155243, -0.05391697585582733, 0.019970625638961792, 0.06352131813764572, -0.05678057298064232, 0.004002926871180534, -0.049995750188827515, 0.05274631455540657, -0.0502006895840168, -0.08883686363697052, -0.07119745761156082, 0.043049927800893784, 0.07067855447530746, -0.017065027728676796, 0.07972317934036255, 0.051466867327690125, 0.02703358605504036, 0.015054886229336262, 0.06858917325735092, 0.0064668357372283936, -0.02666882798075676, -0.05425378307700157, -0.06284839659929276, 0.0049095191061496735, 0.08462942391633987, 0.01851058565080166, 0.027686668559908867, -0.019246963784098625, -0.043507739901542664, -0.041432157158851624, -0.04720885306596756, -0.0016806288622319698, -0.01602541096508503, 0.016466520726680756, -0.06568397581577301, -0.05460387095808983, -0.08376047015190125, -0.010168692097067833, 0.12017899751663208, -0.15379254519939423, -0.0010113657917827368, 0.023721061646938324, -0.016600167378783226, -0.007123927585780621, 0.013696865178644657, 0.06526432931423187, -0.10708923637866974, 0.04896268993616104, 0.09730330109596252, 0.017443524673581123, 0.0298813134431839, 0.037881165742874146, 0.07790081948041916, -0.07626393437385559, 0.04582200199365616, -0.08566936105489731, -0.1284264177083969, 0.019763272255659103, 0.05982929840683937, 0.1308412402868271, 0.10450426489114761, 0.03426438570022583, -0.0876881554722786, 0.01986246556043625, 0.0829804539680481, -0.009395802393555641, 0.02232467010617256, -0.17460641264915466, 0.06762387603521347, 0.04187237471342087, -0.062328752130270004, -0.061231471598148346, -0.01957561820745468, 0.02792799472808838, -0.07542027533054352, 0.02688751183450222, 0.1097843125462532, -0.03704840689897537, -0.10386838018894196, 0.0704495757818222, 0.054309047758579254, -0.03511739894747734, 0.02207741141319275, -0.03936513885855675, 0.04636649042367935, -0.03844611346721649, 0.15117742121219635, -0.026060376316308975, 0.010084512643516064, -0.029549144208431244, 0.020574206486344337, -0.011764737777411938, -0.016229810193181038, 0.016478784382343292, -0.03330899029970169, -0.012583362869918346, 0.0372854620218277, -0.05205903202295303, 0.008976362645626068, 0.05342352017760277, 0.031969379633665085, 0.09827534854412079, 0.11043146252632141, -0.027169711887836456, -0.010608533397316933, -0.05120120942592621, 0.09101647883653641, -0.11159005016088486, -0.07724546641111374, -0.07225504517555237, 0.041119467467069626, -0.02737240493297577, -0.05272366479039192, -0.10393255203962326, 0.05466601625084877, -0.00637579383328557, -0.13789696991443634, 0.03464750200510025, -0.024296429008245468, -0.031112533062696457, 0.022725526243448257, -0.024004923179745674, -0.043744638562202454, -0.0970766544342041, -0.07310277968645096, -0.06509228050708771, 0.03556927666068077, 0.08410539478063583, -0.04411844164133072, -0.03398440405726433, -0.007636806461960077, -0.048086121678352356, -0.08852195739746094, -0.01720789261162281, 0.08601544797420502, -0.10864084213972092, -0.0961635559797287, -0.004106651525944471, 0.04736331105232239, -0.04065787419676781, -0.07672832161188126, 0.08365099132061005, 0.019511064514517784, 0.061487141996622086, -0.008585827425122261, 0.0018319804221391678, 0.08071509003639221, -0.08577054738998413, 0.08193389326334, 0.12552006542682648, -0.08693691343069077, -0.08267959207296371, 0.01833263598382473, 0.08133856952190399, 0.046397581696510315, -0.050849251449108124, 0.11241177469491959, 0.026503846049308777, 0.012011432088911533, 0.005522085353732109, 0.006098815705627203, 0.059678610414266586, 0.10585571825504303, 0.07273048907518387, -0.06244282051920891, -0.0789983794093132, -0.11211004108190536, 0.010095633566379547, 0.11540234088897705, 0.03173184022307396, -0.07766057550907135, -0.03349977359175682, -0.08578387647867203, -0.10970907658338547, -0.04036068543791771, -0.10391103476285934, 0.09862537682056427, 0.06732535362243652, 0.09450694918632507, 0.06004894897341728, 0.08792570978403091, -0.12194088101387024, 0.06981901079416275, -0.00494508258998394, -0.023629749193787575, 0.06159278005361557, 0.12291532009840012, 0.010670226998627186, -0.004062618128955364, -0.05551505833864212, -0.12688012421131134, -0.039926618337631226, 0.026802772656083107, -0.05588807538151741, -0.026840468868613243, 0.13197724521160126, -0.05150333046913147, 0.13255928456783295, 0.053578734397888184, 0.08846738189458847, -0.02907346561551094, -0.0008861100068315864, 0.043048638850450516, 0.07546231895685196, -0.08757161349058151, 0.027773981913924217, 0.04323089122772217, -0.07583461701869965, 0.04148464277386665, 0.007951724343001842, -0.007272812537848949, -0.0451800599694252, -0.03299446031451225, -0.10410665720701218, 0.04216562584042549, 0.09376540035009384, 0.06555589288473129, 0.05723268911242485, -0.01960078254342079, -0.1325644999742508, -0.05892482399940491, -0.08474509418010712, 0.001884097931906581, 0.09261084347963333, -0.0832025483250618, 0.08389569073915482, -0.0938575342297554, -0.035873737186193466, -0.07222186028957367, -0.08463743329048157, 0.10481614619493484, 0.06252484768629074, 0.053628966212272644, -0.059030547738075256, -0.08215326815843582, -0.05966709554195404, -0.014091350138187408, 0.001693548052571714, -0.06810058653354645, -0.10169024020433426, 0.0413576178252697, -0.05847008526325226, 0.03487835079431534, -0.06820749491453171, -0.09960450977087021, -0.06906222552061081, 0.040335748344659805, -0.08656729757785797, -0.031942810863256454, -0.09774470329284668, 0.13521307706832886, 0.020014606416225433, 0.0092628700658679, 0.06780242174863815, -0.10831381380558014, -0.09226881712675095, 0.027826936915516853, -0.044355686753988266, 0.07078159600496292, -0.02046634629368782, 0.048467572778463364, -0.04633687809109688, 0.04063253849744797, -0.003391559701412916, -0.09701764583587646, -0.044596098363399506, -0.004112287424504757, -0.10444166511297226, 0.026694195345044136, -0.08315826952457428, -0.08306949585676193, 0.06917507946491241, -0.02132057584822178, 0.04746847227215767, -0.041015975177288055, -0.010271632112562656, -0.06668466329574585, -0.07042296230792999, -0.029238954186439514, -0.05020995810627937, 0.03312474116683006, -0.15057571232318878, 0.09817426651716232, 0.05125417187809944, 0.12785187363624573, 0.08872876316308975, 0.11683257669210434, -0.022360511124134064, 0.06911887973546982, 0.11976605653762817, 0.03809741511940956, 0.01595962420105934, 0.10770446062088013, -0.08650457113981247, 0.10451242327690125, 0.003277856158092618, -0.13169732689857483, 0.09899202734231949, 0.10835499316453934, 0.03537101298570633, -0.05225808918476105, 0.10754948109388351, 0.035531867295503616, -0.04531748592853546, 0.14529909193515778, 0.007767738774418831, 0.13474398851394653, 0.13712696731090546, 0.12221934646368027, 0.06885286420583725, 0.06317172944545746, -0.026997854933142662, 0.027370993047952652, -0.006835298612713814, -0.10714640468358994, -0.07574371993541718, -0.010435175150632858, -0.0056512244045734406, 0.06568367779254913, 0.014795957133173943, 0.08379393070936203, -0.08288723975419998, 0.06892339140176773, -0.09515301138162613, 0.04800313338637352, 0.05459852144122124, 0.09704891592264175, -0.05034526437520981, 0.036733247339725494, -0.01174239069223404, 0.06059354543685913, 0.05864235758781433, -0.006005175411701202, -0.05009716376662254, -0.10289392620325089, 0.062416817992925644, 0.1282062828540802, 0.08076903223991394, -0.013122023083269596, 0.0034177061170339584, -0.06788726150989532, -0.055542800575494766, -0.023458588868379593, -0.05464181676506996, -0.023145882412791252, 0.004740441218018532, -0.08277338743209839, 0.01486124936491251, -0.07942891865968704, -0.10882824659347534, -0.06958816200494766, 0.03840707615017891, 0.09439937770366669, -0.04842812940478325, 0.07651856541633606, -0.021677903831005096, -0.006779429502785206, 0.11540152877569199, 0.02443389780819416, -0.08672872930765152, -0.04884757846593857, 0.06391792744398117, -0.084181047976017, -0.10453149676322937, -0.07046126574277878, 0.10822991281747818, 0.0217406265437603, -0.03739999607205391, -0.09175694733858109, -0.14669349789619446, 0.05814288184046745, 0.03129367530345917, -0.030790837481617928, 0.07482561469078064, 0.049908120185136795, 0.09558378905057907, 0.04464064911007881, 0.1367727369070053, 0.038664255291223526, 0.029115896672010422, -0.016438327729701996, 0.03144152835011482, -0.003823793726041913, 0.056246839463710785, -0.01667238399386406, 0.12808921933174133, 0.03642986714839935, -0.0124115701764822, -0.0587727390229702, -0.04417561739683151, -0.0723966434597969, -0.08110307902097702, 0.01620127074420452, -0.038515858352184296, 0.0736381933093071, 0.006813909392803907, 0.144139364361763, 0.02458847314119339, 0.03487768769264221, 0.10252729803323746, -0.08126787096261978, -0.07642262428998947, 0.033861175179481506, 0.008150061592459679, -0.021004006266593933, 0.05075399577617645, -0.03999193385243416, 0.019557664170861244, 0.055443257093429565, 0.004142989404499531, 0.05694171041250229, 0.0958637222647667, 0.01910359598696232, 0.08003652840852737, 0.024968240410089493, -0.09701057523488998, 0.016032027080655098, -0.05357357859611511, 0.08550054579973221, 0.059749454259872437, 0.04735974222421646, -0.08301963657140732, 0.09905081242322922, -0.014865421690046787, 0.09980905801057816, 0.03981468454003334, -0.06131935119628906, -0.07049724459648132, 0.058970287442207336, 0.07650653272867203, -0.04970737174153328, -0.06651633977890015, -0.0996704250574112, -0.06085624918341637, 0.022422155365347862, 0.051990725100040436, -0.07056966423988342, -0.04693124443292618, 0.025892850011587143, -0.040222834795713425, 0.071109838783741, -0.018403762951493263, -0.10063225775957108, 0.14351946115493774, 0.023342734202742577, -0.08849979937076569, -0.048608437180519104, 0.04732419177889824, -0.04611003398895264, -0.058420971035957336, -0.06100335717201233, -0.020642230287194252, 0.09693532437086105, 0.0603523887693882, 0.09348125755786896, -0.07531026005744934, 0.0945749431848526, 0.10826239734888077, 0.10421010106801987, -0.07640648633241653, 0.003122597001492977, 0.10486507415771484, -0.02325253002345562, -0.12313946336507797, 0.08745015412569046, -0.06677693128585815, -0.03153197094798088, -0.07058759778738022, 0.11026720702648163, 0.054433491080999374, 0.13341408967971802, 0.08360164612531662, -0.02624167501926422, 0.014721180312335491, -0.02815261296927929, 0.07316094636917114, -0.05255470797419548, 0.061473894864320755, -0.03377803415060043, 0.036212850362062454, 0.0630427673459053, -0.06866202503442764, 0.024662503972649574, -0.01474683079868555, 0.04845164343714714, -0.010956854559481144, -0.11075142025947571, -0.04076739773154259, -0.06880133599042892, -0.08024302870035172, 0.02110917866230011, 0.021741757169365883, 0.06416288018226624, 0.055366791784763336, -0.044101279228925705, -0.03424404188990593, 0.012257060967385769, 0.01479316595941782, -0.06550591439008713, 0.023736629635095596, 0.011227872222661972, -0.0010201487457379699, 0.011327406391501427, -0.08648674935102463, 0.060750048607587814, -0.08855587989091873, -0.005243172403424978, -0.11655894666910172, -0.00660855183377862, 0.014542220160365105, -0.09506828337907791, 0.036480486392974854, 0.022117270156741142, -0.07673445343971252, -0.101728156208992, -0.0063133141957223415, -0.0346832349896431, -0.05214448645710945, -0.014149089343845844, -0.06832053512334824, 0.060395747423172, -0.16194388270378113, 0.05046120658516884, 0.023551300168037415, 0.144208163022995, 0.028646914288401604, 0.06569086760282516, -0.04816150665283203, -0.006096778903156519, -0.03571460768580437, 0.026050761342048645, 0.07686208933591843, 0.1192881241440773, 0.11129648238420486, 0.005696633365005255, -0.07051211595535278, 0.002894424367696047, -0.035914346575737, -0.08537015318870544, -0.10120400041341782, 0.016371915116906166, -0.07334675639867783, -0.03661477938294411, 0.08027077466249466, 0.10630496591329575, -0.03640444576740265, -0.08466017246246338, 0.018357226625084877, -0.0864492878317833, -0.04212896525859833, -0.024422835558652878, 0.10982473194599152, 0.05130802094936371, 0.05413736030459404, 0.05498234182596207, 0.05100419372320175, 0.02749386616051197, -0.07686199992895126, -0.10031820088624954, 0.08483284711837769, -0.026867400854825974, -0.050959426909685135, -0.09648685902357101, -0.0932282954454422, -0.0716681033372879, 0.045992087572813034, -0.052864350378513336, -0.07752825319766998, 0.12441008538007736, 0.0061485618352890015, 0.04258294776082039, -0.060750678181648254, -0.07956632226705551, -0.0021727741695940495, -0.02285795472562313, -0.02249940298497677, -0.10053655505180359, -0.06399713456630707, 0.021264921873807907, -0.04080298915505409, 0.012519286014139652, -0.03302992135286331, -0.08937498927116394, 0.02867073006927967, -0.05347830057144165, 0.010906367562711239, -0.006188721861690283, -0.03786197677254677, -0.037453100085258484, -0.08656031638383865, -0.007323508150875568, -0.026475002989172935, -0.07677662372589111, 0.035959642380476, 0.0408795028924942, 0.13236147165298462, 0.02914869785308838, 0.09561567008495331, -0.05522780120372772, 0.07175835967063904, -0.03152477368712425, 0.04546580836176872, 0.037952400743961334, 0.08681440353393555, -0.08333232998847961, 0.01507297158241272, -0.04401032254099846, -0.13961103558540344, 0.024775277823209763, -0.04596826434135437, 0.05099744722247124, -0.09589247405529022, 0.03598678484559059, -0.089995376765728, 0.047558337450027466, 0.07421854883432388, -0.002807860029861331, -0.08014695346355438, 0.09517353028059006, -0.07576896250247955, 0.00043936099973507226, -0.025169897824525833, 0.0019010512623935938, -0.04703236743807793, 0.09487079083919525, 0.06555449217557907, 0.07268264889717102, 0.1122988909482956, 0.02523944154381752, 0.08299563080072403, 0.10627426952123642, 0.03130857273936272, -0.020344316959381104, 0.08289338648319244, 0.0465950146317482, -0.00729029905050993, -0.05657871067523956, -0.031690843403339386, -0.09458785504102707, 0.03446875512599945, 0.03564247488975525, 0.111052006483078, 0.08743512630462646, -0.06962834298610687, 0.023407842963933945, 0.08289171755313873, -0.08408819884061813, 0.04610075429081917, 0.11106493324041367, 0.10752207040786743, 0.12533661723136902, -0.07069233804941177, -0.07915230840444565, 0.01406614389270544, 0.11695469915866852, 0.07777244597673416, -0.08516401052474976, -0.017970994114875793, 0.07750347256660461, 0.05113096907734871, 0.027082080021500587, -0.11760107427835464, -0.12234333157539368, 0.04199879616498947, 0.03382565453648567, 0.12049781531095505, -0.07279231399297714, 0.017405396327376366, -0.012679357081651688, 0.032246287912130356, -0.03551046922802925, -0.049321822822093964, 0.06803567707538605, 0.05157415196299553, -0.04682726413011551, 0.040118105709552765, 0.060112301260232925, -0.048026133328676224, 0.017472565174102783, 0.0005307148676365614, -0.08696646988391876, 0.051369037479162216, -0.04609199985861778, 0.01777561753988266, 0.019460368901491165, 0.03233496844768524, -0.10337208956480026, -0.09230479598045349, -0.09670443832874298, -0.12329822033643723, 0.0162995383143425, -0.04979785159230232, -0.026522960513830185, 0.017852898687124252, 0.0835314467549324, 0.15662318468093872, 0.056814659386873245, -0.009204274974763393, 0.17949430644512177, 0.057044412940740585, -0.05634845048189163, -0.12656007707118988, -0.10968370735645294, -0.05246293172240257, -0.04494590312242508, 0.08260863274335861, -0.060087382793426514, 0.0015217626933008432, 0.023348378017544746, -0.1003066822886467, 0.009481644257903099, 0.061479490250349045, -0.1275770515203476, 0.03063608519732952, 0.0314275287091732, 0.10729990899562836, -0.046875081956386566, 0.02927635796368122, -0.10419498383998871, -0.029035290703177452, 0.047785744071006775, 0.02662728540599346, -0.03941702842712402, -0.07664745301008224, 0.061000991612672806, -0.05207981541752815, -0.02785375341773033, 0.00040815334068611264, -0.09889800101518631, -0.11639443784952164, -0.0815930962562561, -0.09235845506191254, 0.052244845777750015, -0.05165891721844673, -0.06481105089187622, -0.0018907345365732908, 0.06492459028959274, -0.09643420577049255, 0.08272460848093033, -0.017615554854273796, -0.07541976124048233, -0.03364883363246918, 0.09419483691453934, -0.07532888650894165, 0.07797542959451675, 0.0049995239824056625, -0.019692162051796913, 0.06968212872743607, -0.0412461943924427, 0.06247597560286522, 0.020909404382109642, -0.04057181999087334, -0.1180364266037941, -0.10666940361261368, 0.0314369760453701, -0.008572028018534184, -0.09602953493595123, 0.0022879771422594786, 0.022774694487452507, 0.0037619308568537235, -0.024933988228440285, 0.05583722144365311, -0.0941559374332428, -0.07299380004405975, -0.08756899833679199, -0.14251179993152618, -0.08221247047185898, 0.0028865027707070112, 0.11125046014785767, -0.022456157952547073, 0.06916821748018265, 0.009844240732491016, -0.03677447885274887, 0.0820239707827568, -0.0410122349858284, -0.045730073004961014, 0.058950427919626236, 0.06583785265684128, -0.06785119324922562, 0.010930895805358887, 0.08446060866117477, 0.04442691430449486, -0.00880963634699583, 0.04347878322005272, 0.05951516702771187, -0.06492756307125092, -0.01931016519665718, 0.05676504969596863, -0.011991381645202637, -0.005010063294321299, 0.07482769340276718, -0.08311758190393448, 0.042943015694618225, -0.08842600136995316, -0.05595264956355095, -0.03586617484688759, 0.0018926059128716588, -0.042965780943632126, 0.16852149367332458, 0.001163726206868887, -0.11793302744626999, 0.022799426689743996, -0.0884268656373024, 0.06370694190263748, 0.11236310750246048, 0.1127847284078598, 0.07977904379367828, -0.08041981607675552, 0.08734982460737228, -0.09324897825717926, 0.09971258789300919, -0.09224269539117813, -0.0480709969997406, -0.01409741397947073, -0.05949223041534424, -0.028457608073949814, 0.018776191398501396, 0.01406317949295044, 0.020148564130067825, -0.060626789927482605, 0.00031428446527570486, 0.10679303854703903, -0.014575247652828693, 0.030585527420043945, -0.004227881319820881, -0.06505168974399567, -0.08395969122648239, -0.079381063580513, 0.013417903333902359, 0.010641653090715408, 0.03958377242088318, -0.0950239971280098, -0.008894531056284904, -0.07323089987039566, -0.00498554902151227, -0.1357944756746292, 0.08016374707221985, 0.024934737011790276, -0.11793254315853119, 0.021145278587937355, -0.11830101162195206, 0.08676529675722122, -0.09437233954668045, -0.04954567179083824, -0.05256357789039612, 0.09107475727796555, -0.10302748531103134, 0.08172000199556351, -0.014705651439726353, -0.008544153533875942, -0.08764584362506866, 0.05582208186388016, -0.05633322149515152, -0.04457663744688034, 0.005202991887927055, 0.045450516045093536, -0.03657538816332817, 0.006286715157330036, 0.072922483086586, -0.0357983373105526, 0.06491569429636002, 0.06308121234178543, 0.042621348053216934, 0.1728481501340866, -0.02430676482617855, 0.0845247134566307, 0.043128665536642075, 0.002284200629219413, 0.02131425403058529, -0.05886422097682953, 0.11705540120601654, -0.11024319380521774, -0.10671751946210861, 0.08124370127916336, 0.014954237267374992, 0.062463779002428055, 0.03455129265785217, -0.01734134927392006, 0.039907459169626236, 0.04292235150933266, 0.0353299155831337, -0.09634698927402496, 0.113932766020298, 0.05987897515296936, -0.09486611187458038, 0.02721213921904564, 0.06692282110452652, -0.029507851228117943, 0.09316972643136978, 0.04087385907769203, -0.05464285984635353, -0.008223789744079113, -0.11339487880468369, -0.09686645865440369, 0.1470901072025299, 0.012764643877744675, -0.04945722222328186, -0.024975048378109932, 0.022608647122979164, -0.0072781420312821865, 0.03839585930109024, 0.13265928626060486, 0.007405658718198538, 0.05502443015575409, -0.014141379855573177, -0.0974961668252945, 0.047386810183525085, -0.09331703931093216, 0.05284775048494339, -0.07235753536224365, -0.12079502642154694, 0.025678392499685287, -0.005308354739099741, 0.14298486709594727, 0.01333831250667572, 0.06205267831683159, -0.02596614882349968, -0.008437532931566238, -0.102808877825737, -0.046478062868118286, -0.12145791202783585, 0.06194549426436424, -0.09535953402519226, 0.01937577873468399, -0.05964011698961258, -0.13150790333747864, 0.06435535848140717, 0.10703660547733307, -0.005202687811106443, -0.02762552537024021, 0.11330873519182205, -0.06363608688116074, -0.08404187858104706, -0.012071102857589722, 0.07470538467168808, -0.09872835129499435, -0.02413737215101719, -0.007792087737470865, 0.07520470023155212, -0.03413371369242668, 0.024070613086223602, 0.06828135997056961, 0.058297790586948395, 0.04145774617791176, 0.04084281250834465, 0.045286454260349274, -0.09649115055799484, -0.06288093328475952, 0.015280557796359062, 0.07395216077566147, -0.07592501491308212, -0.056427668780088425, 0.028970835730433464, 0.020526524633169174, 0.028372187167406082, 0.08218933641910553, -0.03200330212712288, -0.07040330767631531, -0.03058953955769539, -0.017250146716833115, -0.05233823508024216, 0.05456193909049034, 0.012451978400349617, -0.11691046506166458, -0.008952274918556213, 0.05939234048128128, -0.09058894217014313, 0.08494237810373306, -0.05844868719577789, -0.04665873944759369, 0.017897043377161026, -0.030807796865701675, 0.06927138566970825, -0.06001792103052139, -0.0585448257625103, -0.11627542972564697, -0.025700051337480545, -0.058717694133520126, 0.09725994616746902, -0.038649335503578186, 0.036149464547634125, -0.05681714415550232, 0.12488625943660736, 0.021750135347247124, 0.036591216921806335, 0.12055911868810654, 0.032138776034116745, 0.015136241912841797, -0.00614120764657855, -0.026514889672398567, -0.1048932820558548, 0.0979466661810875, -0.09823732078075409, -0.08416982740163803, 0.13449335098266602, 0.05213376134634018, -0.05145769566297531, 0.07026808708906174, 0.019618621096014977, -0.036296453326940536, 0.03867937624454498, -0.09742099791765213, 0.01554784458130598, -0.06454972177743912, 0.012397042475640774, 0.09973371773958206, -0.06255906075239182, -0.02464039996266365, -0.11093292385339737, -0.025254689157009125, 0.01732492260634899, -0.0009091036627069116, -0.0009948889492079616, -0.08474376052618027, -0.04674321413040161, 0.058476854115724564, -0.028455248102545738, -0.038338132202625275, 0.11827349662780762, -0.0522085502743721, -0.017046833410859108, -0.03884804621338844, 0.126570463180542, -0.08168169111013412, 0.0073188291862607, 0.08768869936466217, -0.06954402476549149, -0.020525429397821426, -0.0786055326461792, 0.03441657871007919, 0.1232299730181694, -0.07907096296548843, -0.12266724556684494, 0.10582830011844635, 0.10037439316511154, 0.06610478460788727, 0.006905375514179468, -0.07795962691307068, 0.06866064667701721, 0.11073916405439377, 0.08941362053155899, -0.0939931645989418, 0.08856520056724548, -0.022650333121418953, -0.031511444598436356, -0.07157336920499802, -0.017065919935703278, -0.009236033074557781, 0.07023435086011887, 0.08538252115249634, 0.04898252710700035, -0.14287720620632172, -0.0015891824150457978, 0.019577983766794205, -0.00037928158417344093, -0.05338433012366295, -0.07976971566677094, 0.08192769438028336, 0.017677485942840576, 0.07557991147041321, 0.042060498148202896, -0.09308602660894394, 0.12672220170497894, 0.08061620593070984, 0.022771470248699188, -0.06720438599586487, -0.08503729850053787, -0.03716803342103958, 0.011376298032701015, 0.07052299380302429, 0.11020412296056747, -0.0867019072175026, 0.13275454938411713, -0.051438309252262115, -0.05739825963973999, 0.018137194216251373, 0.06169947236776352, -0.05206437036395073, 0.09335966408252716, -0.021193144842982292, -0.06303854286670685, -0.04109078273177147, 0.004252431448549032, 0.0342496782541275, 0.061584845185279846, 0.09103347361087799, -0.07242582738399506, 0.08901146054267883, 0.04412282630801201, 0.028013665229082108, -0.10555560141801834, 0.005542166996747255, 0.07171803712844849, 0.14837296307086945, 0.05748561769723892, 0.1074916198849678, 0.04622908681631088, -0.019140280783176422, 0.025221969932317734, 0.02210194617509842, 0.05083073303103447, -0.021003223955631256, -0.050407011061906815, 0.05374510958790779, 0.0564042367041111, 0.0352584570646286, -0.08728916943073273, -0.08249667286872864, 0.10108083486557007, -0.00891919806599617, -0.022587308660149574, -0.09596335142850876, 0.036689747124910355, 0.07964158058166504, 0.02752341330051422, -0.026155317202210426, -0.011597869917750359, -0.09297560155391693, 0.0017756156157702208, -0.14963243901729584, 0.0066636051051318645, -0.0704064890742302, -0.07629840821027756, 0.03104131482541561, -0.08737291395664215, -0.030631806701421738, 0.01707409881055355, -0.12070205807685852, 0.040285591036081314, 0.05236630141735077, 0.10078992694616318, -0.02142864651978016, 0.07720720022916794, -0.0904727503657341, 0.06321081519126892, 0.06409210711717606, 0.009957795962691307, 0.005236450117081404, 0.03909113258123398, -0.0407278798520565, 0.10205123573541641, -0.06984645873308182, -0.01250926498323679, 0.02737116999924183, -0.016822492703795433, 0.1102662980556488, -0.03877256065607071, -0.030477633699774742, 0.09707146137952805, -0.0867462307214737, 0.08647901564836502, -0.04605879634618759, -0.0469689667224884, -0.019573668017983437, -0.047780316323041916, 0.0323433019220829, -0.10239095985889435, 0.02516339160501957, 0.0737791359424591, 0.032264746725559235, 0.019396428018808365, 0.0056651378981769085, -0.0313538983464241, 0.08991391211748123, 0.13433212041854858, -0.019698038697242737, 0.022720536217093468, 0.030634641647338867, 0.02014298550784588, -0.01690460741519928, 0.028559565544128418, -0.07860494405031204, 0.05717456713318825, 0.09156931191682816, 0.07227369397878647, 0.03102153167128563, -0.012816451489925385, 0.08872107416391373, 0.024618376046419144, 0.15443842113018036, -0.007284439634531736, -0.10184909403324127, -0.08162371814250946, 0.024664102122187614, -0.08306009322404861, -0.04148648679256439, -0.0520939864218235, -0.05345267429947853, 0.03126594051718712, -0.04955856874585152, 0.10450945049524307, 0.055934734642505646, -0.0523570254445076, 0.03241625800728798, 0.10270030051469803, -0.0009467352647334337, -0.1046198159456253, -0.10899893194437027, 0.05812014639377594, 0.0715983510017395, -0.01676880195736885, 0.06926994770765305, 0.006098069250583649, -0.06812665611505508, -0.04417767375707626, 0.09209907799959183, 0.09069329500198364, -0.14567773044109344, 0.07988356053829193, -0.024152444675564766, 0.12650702893733978, -0.05732409283518791, -0.08480063080787659, 0.0923837423324585, 0.06827419996261597, -0.11548884212970734, 0.04445774108171463, -0.017557553946971893, 0.07886294275522232, -0.0197881031781435, -0.0843258872628212, 0.06439488381147385, -0.0867333635687828, -0.12212864309549332, -0.025979194790124893, -0.052062153816223145, 0.05131780356168747, -0.09581593424081802, -0.0065497541800141335, -0.010324065573513508, -0.005606420338153839, -0.0901978611946106, -0.040101636201143265, -0.0632987767457962, -0.08354143053293228, 0.09907683730125427, 0.1476386934518814, -0.10197783261537552, 0.020233886316418648, 0.049530189484357834, 0.0824127048254013, -0.039271846413612366, 0.05394001305103302, 0.056650351732969284, -0.027789834886789322, -0.000914292992092669, -1.930887628986966e-05, -0.0036445106379687786, -0.07905437052249908, -0.04653970152139664, 0.08078308403491974, -0.0470375120639801, -0.03976752236485481, -0.09050550311803818, -0.14513914287090302, 0.04306616634130478, -0.1043507382273674, 0.0016788545763120055, -0.06696907430887222, -0.09193931519985199, 0.0007296368712559342, 0.017978467047214508, -0.09485304355621338, 0.10199158638715744, 0.061097051948308945, 0.004785002209246159, 0.11577777564525604, 0.04785668104887009, 0.021113116294145584, 0.02337963692843914, -0.027987182140350342, 0.06624731421470642, -0.14115507900714874, 0.0034849646035581827, -0.04588685184717178, -0.037078648805618286, -0.08900291472673416, -0.008663567714393139, 0.05143681541085243, 0.08761937916278839, -0.06490116566419601, -0.1031128317117691, 0.013334301300346851, 0.04004625231027603, -0.10944352298974991, 0.06703797727823257, -0.04021394997835159, -0.039754752069711685, 0.06819859147071838, 0.023847535252571106, 0.12283355742692947, 0.04721890762448311, -0.06550464779138565, -0.10124494135379791, -0.022740662097930908, -0.09932295233011246, 0.05244901403784752, -0.14846697449684143, 0.10702508687973022, 0.027530692517757416, 1.85531098395586e-05, -0.01269648689776659, 0.018831999972462654, 0.00873325951397419, -0.037999093532562256, 0.13944043219089508, 0.07936820387840271, 0.11256732791662216, -0.030045494437217712, -0.0535813644528389, -0.0529957190155983, 0.07388518005609512, 0.02199367806315422, -0.0061781397089362144, 0.0013052503345534205, -0.08978395164012909, -0.05292734131217003, 0.022838521748781204, -0.059107545763254166, 0.15640169382095337, -0.11375368386507034, -0.03525378927588463, 0.0021212769206613302, -0.022891301661729813, -0.015618197619915009, 0.017121126875281334, -0.024288523942232132, 0.12802155315876007, -0.03932439535856247, 0.0017030651215463877, 0.10531210154294968, -0.0832279697060585, 0.0057406947016716, -0.08860524743795395, -0.05817916616797447, -0.02643292024731636, -0.07161984592676163, -0.07118167728185654, 0.11060851067304611, 0.060651347041130066, 0.012388009577989578, -0.05235333368182182, 0.11273662000894547, -0.02933296374976635, -0.0512114055454731, 0.09041973203420639, 0.15456852316856384, 0.0746297538280487, 0.07664333283901215, -0.04449190944433212, -0.05244488641619682, 0.06980333477258682, -0.03000861220061779, -0.12581169605255127, -0.06926679611206055, -0.05166829004883766, 0.08667422086000443, 0.05328146368265152, 0.08718371391296387, 0.026836825534701347, -0.11223083734512329, 0.09398599714040756, 0.025815673172473907, -0.04102747514843941, -0.05734970048069954, -0.09190057218074799, -0.017355360090732574, -0.11584658920764923, 0.024671269580721855, -0.029983917251229286, 0.028466902673244476, -0.07962073385715485, -0.02688675932586193, -0.04355807229876518, -0.007172199431806803, 0.07846422493457794, 0.03469309210777283, 0.10085971653461456, -0.00891704112291336, -0.04225250706076622, -0.04925563558936119, 0.0908251404762268, 0.1092110201716423, -0.02048538625240326, -0.03892408311367035, 0.034691598266363144, 0.05747297778725624, 0.08867014199495316, 0.006267532706260681, -0.11750485748052597, 0.0623505562543869, 0.08005479723215103, 0.04250016808509827, 0.0482962392270565, 0.04878886044025421, -0.05289959907531738, 0.07831394672393799, 0.02056061290204525, -0.06314319372177124, -0.013317438773810863, -0.020438218489289284, 0.053347185254096985, -0.11833052337169647, -0.08338861167430878, -0.08391322195529938, -0.03154565766453743, 0.0628243088722229, 0.075157031416893, -0.007570330519229174, -0.010708533227443695, -0.017022086307406425, -0.02711816504597664, -0.05736168846487999, -0.0982852652668953, 0.01586395502090454, 0.08602675795555115, -0.12134143710136414, -0.06448569893836975, 0.07986723631620407, 0.04500826820731163, -0.013230044394731522, -0.08651651442050934, -0.03773215413093567, -0.005728018004447222, 0.012641152366995811, 0.03140654414892197, -0.0038904501125216484, 0.14170940220355988, 0.0387628898024559, 0.09226343035697937, 0.008172158151865005, 0.07793133705854416, 0.011834214441478252, 0.008537974208593369, 0.05326110124588013, 0.00016443735512439162, -0.09152158349752426, -0.10153979063034058, 0.13238245248794556, 0.10866980999708176, -0.018888253718614578, 0.07039765268564224, -0.03574681654572487, -0.09700120985507965, 0.11346781998872757, 0.019004205241799355, 0.09891871362924576, 0.01820836029946804, 0.005442304071038961, 0.021521348506212234, -0.09108181297779083, 0.06916443258523941, -0.033163294196128845, 0.0393955260515213, -0.09771887958049774, -0.09382179379463196, 0.09303488582372665, 0.09856689721345901, -0.05667240917682648, 0.02764422819018364, -0.04356217384338379, 0.025351619347929955, 0.07536397874355316, 0.07951448112726212, 0.044513601809740067, -0.048341285437345505, 0.11110275238752365, 0.0032235069666057825, -0.05580613762140274, -0.058070626109838486, -0.0878303050994873, 0.08203167468309402, 0.09667960554361343, 0.0665149837732315, -0.1313941776752472, -0.027858415618538857, -0.08332464843988419, -0.05517946928739548, 0.022768907248973846, -0.049864284694194794, -0.09096817672252655, -0.0984971672296524, -0.0052130864933133125, 0.06674978882074356, 0.020318716764450073, 0.08145733177661896, -0.018532192334532738, 0.042423248291015625, -0.08641062676906586, 0.08662252128124237, -0.04735404625535011, -0.017286324873566628, -0.12757906317710876, 0.00929320976138115, -0.028510259464383125, 0.005521018523722887, 0.08091782033443451, 0.07531344145536423, -0.0647987574338913, 0.07276206463575363, -0.07472485303878784, -0.13536518812179565, 0.09297721087932587, -0.050877466797828674, -0.08598978072404861, 0.03869481012225151, 0.014438602142035961, 0.06426640599966049, 0.045249566435813904, -0.0253569558262825, -0.005351048428565264, -0.006684140767902136, 0.04254722595214844, -0.012561816722154617, -0.09522326290607452, -0.06468509882688522, 0.061404623091220856, -0.10574786365032196, -0.09944605827331543, 0.09724366664886475, 0.07468962669372559, -0.07930556684732437, 0.07736172527074814, -0.031154954805970192, -0.04921306297183037, 0.04027153551578522, -0.08256231248378754, -0.03551805019378662, 0.023796219378709793, -0.13705132901668549, 0.0668608546257019, 0.07481015473604202, 0.03627641126513481, -0.05745838209986687, -0.02986922673881054, 0.011227958835661411, -0.09021002799272537, 0.08187653869390488, 0.0762290358543396, 0.015251433476805687, -0.015758059918880463, -0.033133216202259064, -0.0004331147065386176, -0.042434267699718475, 0.03905181214213371, 0.06597329676151276, 0.03608126565814018, 0.053008511662483215, -0.06647159159183502, -0.03217163309454918, -0.1120135560631752, 0.010848010890185833, 0.054891616106033325, -0.039127692580223083, 0.01932712458074093, -0.0008092729840427637, 0.0321616567671299, -0.051770806312561035, 0.017236772924661636, -0.06454749405384064, 0.08996246010065079, -0.09913329035043716, 0.0033495852258056402, 0.10514896363019943, 0.014392685145139694, -0.05755927041172981, 0.09321635961532593, -0.04581568017601967, 0.012025089003145695, 0.05621165782213211, 0.005080986302345991, -0.028127992525696754, -0.05605825036764145, -0.02388269640505314, 0.06309650093317032, -0.12483067065477371, 0.035244714468717575, 0.02942662313580513, 0.07910865545272827, -0.10484587401151657, -0.035166241228580475, 0.01859262026846409, -0.10516457259654999, 0.03318256512284279, -0.07728823274374008, -0.06295877695083618, -0.04835192486643791, 0.0847814753651619, -0.0358545184135437, 0.06586375832557678, 0.009266303852200508, 0.02694619446992874, 0.01077096164226532, -0.031043928116559982, 0.03566513583064079, -0.04797615110874176, 0.07809025049209595, -0.083076611161232, 0.06183895841240883, -0.04350937902927399, 0.04525550454854965, 0.018273713067173958, 0.0439479760825634, 0.101295605301857, -0.016822710633277893, -0.0932542160153389, 0.11003229767084122, 0.03808119148015976, 0.026957804337143898, -0.04465813934803009, -0.08083910495042801, 0.026340164244174957, 0.062036000192165375, -0.001108192140236497, -0.06286681443452835, -0.027526848018169403, -0.08333463221788406, -0.03435191512107849, -0.04531341418623924, 0.0520816333591938, -0.10029378533363342, 0.13045848906040192, 0.023150676861405373, -0.07747665047645569, -0.006994740571826696, -0.12306446582078934, 0.03376517817378044, -0.11385231465101242, 0.03837097808718681, -0.13123394548892975, 0.06679268181324005, 0.0362975150346756, -0.10103116184473038, 0.056266900151968, -0.056373368948698044, -0.037070535123348236, -0.08272834122180939, 0.1071329116821289, -0.1729559749364853, 0.06526236981153488, -0.017588961869478226, 0.03801967203617096, 0.004237788263708353, -0.029016884043812752, 0.022780446335673332, -0.07474344968795776, 0.10901986807584763, -0.02777354046702385, 0.06370298564434052, -0.11812669783830643, -0.14294354617595673, -0.035563092678785324, 0.03939402103424072, 0.00012766252621077, -0.07148832827806473, -0.011049682274460793, -0.042949870228767395, 0.029964707791805267, 0.09607341140508652, 0.07925169169902802, 0.04316139221191406, 0.16322581470012665, -0.07197447121143341, -0.05300535634160042, 0.07031318545341492, -0.08109291642904282, 0.012740417383611202, -0.1433054655790329, -0.038894809782505035, -0.04129650816321373, 0.06486506760120392, -0.06496088951826096, 0.07176655530929565, 0.12742604315280914, -0.015926435589790344, -0.12598158419132233, 0.06928551197052002, -0.02264518104493618, 0.17004962265491486, -0.00987447239458561, -0.06286676973104477, 0.0823810026049614, 0.0035829630214720964, 0.02717762440443039, 0.07941284030675888, 0.09855722635984421, -0.026062628254294395, -0.04627865552902222, -0.13905878365039825, 0.1357981264591217, -0.040001221001148224, 0.0675031766295433, 0.027942337095737457, 0.07155279815196991, 0.12314209342002869, 0.08347812294960022, 0.1953759491443634, -0.12844067811965942, -0.01173689216375351, -0.14905716478824615, 0.09319061040878296, -0.06051880121231079, -0.05807976797223091, -0.01946578547358513, 0.028510749340057373, -0.1587681621313095, 0.07652073353528976, -0.15363158285617828, -0.1402202844619751, -0.08335038274526596, 0.11858252435922623, 0.03567514941096306, -0.10442421585321426, 0.10968101024627686, -0.0939217284321785, -0.060695793479681015, -0.07967925816774368, 0.09949488937854767, 0.010422246530652046, 0.02220357395708561, 0.019094038754701614, 0.013893740251660347, 1.1973679647780955e-06, 0.056006986647844315, -0.06768610328435898, 0.03794442489743233, -0.06198602169752121, -0.07835441082715988, 0.1130882129073143, -0.09691715240478516, -0.12519963085651398, 0.08944933116436005, 0.03953833878040314, 0.0076601686887443066, -0.22705848515033722, -0.0227525494992733, -0.11594650894403458, 0.046959418803453445, -0.12136196345090866, -0.010659098625183105, 0.033834341913461685, 0.13708266615867615, -0.026262734085321426, 0.03945428505539894, -0.03619524836540222, -0.07147646695375443, -0.16713246703147888, -0.008038514293730259, 0.03578115254640579, -0.06381771713495255, 0.043033894151449203, -0.0667370855808258, -0.07533912360668182, -0.041393157094717026, 0.05734153091907501, -0.02079116739332676, -0.018164318054914474, -0.029193557798862457, -0.027369676157832146, 0.05544723942875862, -0.04012615233659744, -0.08396169543266296, -0.03300454467535019, 0.04284391179680824, 0.06744275242090225, 0.012814906425774097, 0.16426250338554382, -0.043446362018585205, 0.04821372032165527, 0.06069769710302353, -0.09084240347146988, 0.004387539811432362, -0.10326214879751205, -0.08664177358150482, 0.06719078123569489, -0.02248259447515011, -0.15750253200531006, 0.07169826328754425, 0.012810612097382545, -0.10876336693763733, -0.0938369631767273, 0.01400282047688961, -0.03312167897820473, -0.10315009206533432, -0.07593006640672684, -0.0012713551986962557, 0.10277300328016281, 0.07066598534584045, 0.16685451567173004, 0.006433979142457247, 0.07381134480237961, 0.05918223783373833, 0.05450107902288437, 0.020494507625699043, 0.21154610812664032, -0.22650349140167236, 0.03588056191802025, -0.007786825764924288, 0.14891313016414642, -0.19097085297107697, -0.04448149725794792, 0.07820618152618408, -0.0271401759237051, -0.10985832661390305, -0.0077324360609054565, -0.004017080180346966, -0.024776173755526543, -0.08512790501117706, 0.16260842978954315, -0.13583417236804962, -0.1840939074754715, -0.11118543148040771, -0.04913563281297684, -0.1316608488559723, -0.004526603035628796, -0.09705609828233719, -0.11862694472074509, -0.012436033226549625, -0.1009959727525711, 0.10863102972507477, -0.1420433521270752, 0.14430178701877594, -0.07080678641796112, -0.16634142398834229, -0.05718583986163139, 0.09493746608495712, -0.13213926553726196, -0.15617769956588745, -0.02943349815905094, -0.09278542548418045, 0.06226964667439461, 0.06469359248876572, -0.06496027112007141, -0.1599239856004715, 0.13266003131866455, -0.17778193950653076, -0.09676085412502289, -0.0017905322602018714, -0.061437517404556274, 0.0655503198504448, 0.09800763428211212, 0.11939816176891327, -0.1727714240550995, -0.12849831581115723, -0.05598706007003784, 0.038311149924993515, 0.08150473237037659, -0.1762455254793167, 0.07416404038667679, 0.19500496983528137, 0.053735584020614624, 0.07446140795946121, -0.18003441393375397, 0.09418834745883942, -0.0018892170628532767, -0.09598837047815323, -0.12503305077552795, -0.13325341045856476, -0.054312676191329956, 0.02283637598156929, -0.000417110335547477, 0.14020177721977234, -0.13760413229465485, -0.16415399312973022, 0.1416161060333252, -0.05384978652000427, 0.006793709006160498, 0.04632740095257759, 0.033066276460886, 0.1906624436378479, -0.060194019228219986, 0.19126173853874207, 0.1397484540939331, -0.02840631827712059, -0.10221181809902191, 0.08218972384929657, -0.008331367745995522, -0.12999528646469116, 0.04706190526485443, 0.04726267606019974, 0.09019219875335693, 0.09840740263462067, 0.17014102637767792, -0.01591194048523903, -0.10016365349292755, 0.051201146095991135, -0.04569390416145325, 0.126522034406662, 0.0703425332903862, -0.004772636108100414, 0.21070267260074615, 0.02453397586941719, 0.20182953774929047, 0.06383597105741501, -0.042788293212652206, -0.20313085615634918, -0.01860055699944496, 0.15383556485176086, 0.12281406670808792, 0.15624858438968658, -0.0505281426012516, -0.1108931377530098, 0.12362166494131088, -0.0005968718323856592, -0.015731366351246834, 0.005185415968298912, -0.049763862043619156, 0.04589061439037323, 0.10392941534519196, 0.07833128422498703, -0.08910875767469406, 0.0385916531085968, -0.03722180053591728, -0.03986096754670143, -0.14657925069332123, -0.007669553626328707, 0.15779103338718414, 0.031410735100507736, -0.1414167881011963, -0.08714810758829117, -0.020693808794021606, -0.08205103129148483, -0.05156414955854416, 0.03343099355697632, 0.08573956787586212, -0.09468761831521988, 0.017341408878564835, -0.0014629465295001864, -0.05978364124894142, -0.07844330370426178, -0.13084954023361206, -0.019793912768363953, 6.7566754296422e-05, -0.01069388072937727, 0.051039181649684906, 0.11620163917541504, 0.061183929443359375, 0.04930300638079643, -0.009237180463969707, -0.12842732667922974, -0.04167463630437851, -0.07871667295694351, -0.03973224386572838, -0.10225972533226013, 0.07390618324279785, 0.013524137437343597, 0.03752942383289337, 0.06683450192213058, -0.022428827360272408, -0.017693711444735527, 0.09998657554388046, -0.1595846563577652, -0.030254092067480087, -0.10657868534326553, 0.03370457887649536, -0.08319976180791855, -0.07541953772306442, -0.09826532751321793, -0.07699228823184967, 0.08928264677524567, 0.023668909445405006, -0.2252817004919052, -0.10487374663352966, 0.0035725231282413006, 0.16488522291183472, -0.0547378845512867, -0.03950467333197594, 0.13183628022670746, -0.14360415935516357, 0.1458442211151123, -0.14737136662006378, -0.035917993634939194, -0.16854043304920197, -0.07180298864841461, -0.0030167456716299057, 0.004007328767329454, -0.06549741327762604, -0.09018513560295105, -0.055215559899806976, -0.008937259204685688, 0.09317587316036224, -0.1078076884150505, 0.06971950829029083, -0.0455959215760231, 0.07688184082508087, 0.07465387135744095, 0.1088423877954483, 0.05120617523789406, 0.13472969830036163, 0.029943112283945084, -0.08046161383390427, 0.017047982662916183, -0.06109943985939026, 0.06312475353479385, 0.06864802539348602, 0.06570131331682205, 0.017171794548630714, 0.07930078357458115, 0.14360374212265015, 0.02462604269385338, 0.1624249964952469, 0.15045391023159027, -0.12971530854701996, 0.0051662009209394455, 0.06730113923549652, 0.10674687474966049, -0.07982494682073593, -0.06947500258684158, 0.11151263117790222, -0.04298330843448639, -0.06431407481431961, -0.02969641424715519, 0.01209860946983099, 0.13439048826694489, 0.0652306005358696, 0.0728188157081604, 0.08775823563337326, 0.01871098205447197, 0.13706521689891815, 0.07073219865560532, -0.10702792555093765, 0.07883018255233765, 0.13713642954826355, -0.11109970510005951, 0.025137916207313538, 0.1496155709028244, -0.06530370563268661, -0.057691752910614014, 0.07287919521331787, -0.034046221524477005, 0.043817054480314255, -0.11846644431352615, -0.14311499893665314, 0.11477483063936234, 0.023342961445450783, 0.11356920003890991, 0.014933526515960693, 0.09667503833770752, 0.11349885910749435, 0.11861813813447952, -0.160452201962471, 0.11033401638269424, -0.1730933040380478, -0.039112985134124756, 0.025502072647213936, 0.004298131912946701, -0.1410266011953354, 0.1706036776304245, 0.10900292545557022, 0.0582074336707592, 0.007615436799824238, 0.1053742989897728, 0.17544925212860107, 0.1384090781211853, -0.11778569221496582, 0.19982467591762543, 0.0986521989107132, 0.1467469185590744, -0.05539336055517197, -0.1065765917301178, -0.053424935787916183, -0.011649737134575844, 0.052979856729507446, 0.05221710726618767, -0.10130621492862701, -0.08763913065195084, -0.04987875372171402, -0.17285709083080292, 0.11494281142950058, 0.054690733551979065, -0.057081058621406555, -0.06584828346967697, -0.09036555141210556, -0.05732346326112747, 0.17658059298992157, -0.13005968928337097, -0.054871566593647, -0.13574247062206268, 0.0033563650213181973, 0.1535521149635315, -0.011876293458044529, -0.11855487525463104, -0.12580741941928864, 0.041910722851753235, 0.06352633237838745, -0.021903645247220993, -0.07136481255292892, 0.02661055698990822, 0.034784089773893356, -0.04842193424701691, 0.13541610538959503, 0.013691154308617115, -0.17899523675441742, 0.023105449974536896, -0.10968411713838577, -0.02110164426267147, -0.11596345901489258, 0.1079641580581665, 0.15408261120319366, 0.07470352947711945, 0.026458239182829857, 0.09867962449789047, 0.017138002440333366, 0.09803766757249832, 0.028159819543361664, -0.08209241181612015, 0.0162005927413702, 0.09121619910001755, -0.03383701294660568, 0.0893365815281868, -0.0631905198097229, 0.000677076808642596, -0.049054522067308426, 0.13309618830680847, 0.04737400636076927, -0.07526546716690063, -0.10505300015211105, -0.032478541135787964, 0.14012914896011353, -0.05871008709073067, -0.046717569231987, 0.07005678117275238, 0.06151914969086647, 0.12478513270616531, 0.001445988193154335, -0.08531291782855988, 0.1609409898519516, -0.18911121785640717, 0.09778480976819992, -0.05558525770902634, 0.0650787428021431, -0.016237223520874977, 0.05470546707510948, 0.12598277628421783, 0.02802252024412155, -0.16248591244220734, 0.0987313985824585, 0.04178589582443237, 0.007733616512268782, -0.07336577028036118, -0.04375770688056946, 0.01789715886116028, -0.13291339576244354, -0.02740972675383091, -0.05874671787023544, 0.045480795204639435, 0.12083276361227036, -0.12817542254924774, -0.021219873800873756, -0.02183796465396881, 0.003666905453428626, 0.15013255178928375, -0.06671614199876785, 0.13267305493354797, -0.053304050117731094, -0.007146780379116535, -0.09899351000785828, -0.10869867354631424, 0.008177539333701134, -0.08119966834783554, 0.06325482577085495, -0.06877515465021133, -0.009134693071246147, -0.050790444016456604, 0.014886974357068539, 0.0028221583925187588, 0.04206331819295883, -0.014766099862754345, -0.06778650730848312, 0.06737606972455978, -0.0810316801071167, -0.1473933458328247, -0.15614427626132965, -0.03666098415851593, -0.0802140012383461, -0.09474854171276093, -0.14261846244335175, 0.008836275897920132, 0.10926253348588943, -0.054419972002506256, 0.10642116516828537, 0.15090768039226532, 0.17805127799510956, 0.11262095719575882, 0.06484404951334, 0.06540429592132568, 0.002861892804503441, -0.12103182822465897, -0.032083623111248016, -0.0036731224972754717, 0.05956900864839554, -0.06857144832611084, 0.08060611039400101, 0.06026214361190796, 0.044778972864151, 0.07144054025411606, 0.10597754269838333, 0.08087155222892761, -0.08853727579116821, 0.062438271939754486, 0.0799562856554985, 0.01696694828569889, -0.11425293236970901, -0.1371828019618988, -0.07683228701353073, -0.06876284629106522, 0.03526357188820839, -0.027947653084993362, 0.01532263495028019, 0.12743933498859406, -0.01759372092783451, -0.13032478094100952, -0.04712677001953125, -0.01677793636918068, -0.0591336265206337, -0.05781526491045952, -0.02788512408733368, 0.05978187546133995, 0.083596371114254, -0.03564460575580597, 0.19867591559886932, -0.041438762098550797, -0.0721135139465332, -0.02852362021803856, 0.13457055389881134, 0.04189135134220123, 0.14343991875648499, 0.07360035926103592, -0.10328521579504013, 0.09411178529262543, 0.010573683306574821, -0.0948207825422287, 0.030696071684360504, -0.08051671087741852, -0.012974298559129238, 0.03530549630522728, 0.13548916578292847, 0.047486647963523865, 0.1220201775431633, -0.06307268142700195, 0.052910417318344116, -0.0768701359629631, -0.00638447655364871, -0.04132027179002762, -0.1038641706109047, 0.09584229439496994, 0.11931587010622025, 0.06410811841487885, 0.04152766242623329, -0.1434379518032074, 0.019779911264777184, 0.016510440036654472, 0.06006202846765518, -0.05057700350880623, -0.10383891314268112, -0.06331446766853333, 0.22737722098827362, 0.003677086206153035, -0.03300957381725311, 0.07266869395971298, 0.0389477014541626, 0.1315438151359558, 0.029222365468740463, -0.06077120825648308, 0.13167455792427063, -0.09061594307422638, -0.08659510314464569, -0.19686800241470337, -0.09648428857326508, 0.0318886861205101, -0.11851544678211212, -0.04465655982494354, -0.04572201147675514, -0.05666297674179077, 0.06586892902851105, 0.1278257817029953, -0.005624629091471434, 0.11047699302434921, 0.04836650937795639, 0.08282959461212158, 0.054517850279808044, -0.14336912333965302, -0.03304354101419449, 0.061856068670749664, 0.09223126620054245, -0.08533038944005966, -0.07609190046787262, 0.1395668089389801, 0.030267609283328056, 0.032538238912820816, -0.0017404846148565412, 0.1203102245926857, -0.06800668686628342, -0.13690459728240967, -0.14363868534564972, -0.0777527317404747, -0.020125268027186394, -0.026115572080016136, 0.1353074163198471, -0.147247776389122, 0.012113263830542564, -0.02747943624854088, 0.10069408267736435, -0.09273261576890945, -0.058084748685359955, 0.022182175889611244, -0.13907626271247864, 0.11319855600595474, -0.11496333032846451, 0.05371369794011116, -0.032559096813201904, -0.07818189263343811, 0.0834154486656189, 0.07886888831853867, -0.020803606137633324, 0.13224820792675018, 0.002732428489252925, 0.03925584629178047, 0.03959162160754204, 0.03630697354674339, -0.08186651766300201, -0.11615018546581268, -0.019312264397740364, 0.004932267591357231, -0.11501273512840271, 0.028357427567243576, 0.07966960966587067, 0.006024420727044344, 0.03794431686401367, -0.05356171354651451, 0.10966240614652634, -0.09075281023979187, 0.04895782470703125, -0.07078385353088379, 0.056035228073596954, 0.10955967754125595, -0.02423178404569626, -0.026614973321557045, -0.1050872728228569, -0.0030496998224407434, -0.06459113955497742, 0.06718456745147705, -0.022378316149115562, -0.026158234104514122, -0.13429923355579376, 0.042665738612413406, 0.09822511672973633, 0.1822427660226822, -0.037733934819698334, 0.018730752170085907, 0.012065566144883633, -0.042115919291973114, 0.1611114889383316, 0.04063761234283447, -0.10351157188415527, -0.016487209126353264, 0.10592091828584671, -0.06854628026485443, 0.008720364421606064, -0.09337157011032104, -0.02902487851679325, 0.05983096733689308, -0.02700163796544075, -0.01746245101094246, 0.12008623778820038, -0.13636627793312073, 0.1125154048204422, 0.11587975919246674, -0.02997635118663311, -0.06547518819570541, 0.022253142669796944, 0.07999542355537415, 0.14302384853363037, -0.06481634825468063, 0.05295541137456894, 0.014604865573346615, -0.081183061003685, 0.07260680943727493, 0.017027689144015312, 0.0023474767804145813, -0.09199286252260208, 0.008016020990908146, -0.1602391004562378, 0.1603105515241623, -0.16165375709533691, -0.1598738133907318, 0.09216585755348206, 0.04719382897019386, -0.0538739375770092, 0.0425974503159523, -0.027158914133906364, -0.17845259606838226, 0.03691639006137848, -0.08494526147842407, -0.053673096001148224, 0.07051493972539902, 0.12815135717391968, 0.04424894228577614, 0.1934228241443634, 0.09056123346090317, 0.0034507072996348143, 0.14508797228336334, -0.10578428953886032, 0.038173578679561615, 0.04626065492630005, 0.018749797716736794, 0.09163858741521835, 0.0666070356965065, 0.0005882908590137959, -0.0015846530441194773, 0.09673700481653214, -0.024135729297995567, -0.007852070033550262, 0.022359637543559074, -0.1852806806564331, -0.1293097883462906, 0.13546119630336761, -0.1150646060705185, -0.07892974466085434, -0.09852296113967896, -0.14447584748268127, -0.09554019570350647, -0.005287952721118927, 0.06966821849346161, -0.041310686618089676, -0.05497582629323006, 0.1278245598077774, 0.13082411885261536, 0.13790263235569, -0.06678575277328491, 0.14312636852264404, -0.00721959350630641, 0.04220448434352875, 0.045806217938661575, -0.14539854228496552, -0.12433240562677383, -0.20717370510101318, 0.13451583683490753, -0.12477996945381165, -0.019006188958883286, -0.12504485249519348, 0.04438278451561928, 0.10699030756950378, 0.11291414499282837, -0.04975903406739235, -0.023508330807089806, -0.018992654979228973, -0.05589761957526207, -0.010144907981157303, 0.008908660151064396, 0.05854649469256401, -0.08728982508182526, -0.19680117070674896, -0.08050497621297836, -0.06700922548770905, 0.1325531303882599, 5.783563028671779e-05, 0.08878852427005768, -0.02889052778482437, 0.10312002152204514, 0.18272046744823456, -0.05369343236088753, 0.01838304102420807, 0.01924298331141472, 0.05304868891835213, -0.042358096688985825, 0.02505956031382084, 0.045040782541036606, -0.0015227493131533265, 0.17567837238311768, -0.04511828348040581, 0.0730200782418251, -0.05336935818195343, 0.17686375975608826, 0.04757490009069443, -0.1401045024394989, -0.01564335823059082, -0.03964830935001373, -0.0316452756524086, -0.021290868520736694, 0.02474777214229107, -0.1522182673215866, -0.04698603227734566, 0.06991439312696457, 0.1607118546962738, 0.007906637154519558, -0.03524644672870636, 0.04639068990945816, -0.03561415523290634, 0.05237935483455658, -0.063625767827034, -0.054219525307416916, -0.15151673555374146, 0.0055573550052940845, -0.12387528270483017, 0.019881919026374817, 0.050369106233119965, -0.17465345561504364, 0.06668351590633392, 0.02430490218102932, -0.09225884824991226, 0.06552346050739288, 0.10932757705450058, 0.07643250375986099, 0.14384165406227112, 0.025647686794400215, -0.019518237560987473, 0.0707007497549057, -0.0904693529009819, 0.06786394864320755, -0.07543862611055374, 0.08904828876256943, -0.17584308981895447, 0.11602643132209778, 0.07454512268304825, 0.05246862396597862, -0.07041064649820328, -0.04353786259889603, -0.023852890357375145, -0.12463533878326416, 0.12594933807849884, -0.008767168037593365, -0.11042109876871109, 0.05349578335881233, -0.08489671349525452, -0.07871949672698975, 0.015664193779230118, 0.0074986983090639114, -0.0321153961122036, 0.13041575253009796, 0.013805192895233631, -0.027011288329958916, 0.08044805377721786, -0.11520848423242569, 0.10882747173309326, -0.11639699339866638, -0.156428724527359, 0.08515218645334244, -0.02724870666861534, 0.08886756002902985, 0.18813791871070862, -0.04398619383573532, 0.05906889587640762, -0.12928615510463715, 0.0668509230017662, -0.08998338133096695, -0.013209670782089233, -0.14409853518009186, -0.08793315291404724, 0.13428758084774017, 0.1602792739868164, 0.03452087566256523, -0.061643581837415695, 0.03291920945048332, 0.02697981521487236, -0.05988604202866554, -0.03790075704455376, -0.07562638074159622, -0.052906446158885956, 0.015579611994326115, -0.17202326655387878, -0.034948185086250305, -0.09516394138336182, 0.12042121589183807, -0.009836116805672646, -0.041845813393592834, 0.07758398354053497, 0.03842851519584656, 0.07479855418205261, 0.10511855036020279, -0.12134666740894318, -0.08222603052854538, -0.035758525133132935, 0.04773532599210739, 0.04079379886388779, -0.03919451683759689, -0.08779063820838928, -0.04523048177361488, 0.1539149433374405, 0.02017204649746418, -0.020561669021844864, 0.01207462977617979, 0.04432658851146698, 0.15563388168811798, 0.07153184711933136, 0.04152993857860565, -0.09067322313785553, -0.09967119991779327, 0.057306382805109024, 0.0037724131252616644, -0.001248087384738028, -0.02077292464673519, -0.031296346336603165, -0.02197323553264141, 0.18697375059127808, 0.03454992547631264, -0.01918143406510353, -0.057770080864429474, -0.05633191391825676, -0.053315628319978714, -0.04545622691512108, -0.05153312534093857, -0.03517238795757294, 0.1333506852388382, -0.061181385070085526, 0.024905279278755188, -0.035080574452877045, -0.01585734635591507, -0.17089001834392548, -0.0877021998167038, -0.06239365413784981, -0.08508332073688507, 0.07165288925170898, -0.13414546847343445, -0.01070783007889986, 0.05456755310297012, -0.09682025015354156, -0.13616196811199188, 0.035296421498060226, 0.04526006057858467, 0.14854437112808228, -0.11745884269475937, -0.09966613352298737, 0.018546823412179947, -0.013788587413728237, 0.13522982597351074, -0.02963782474398613, -0.021181032061576843, 0.1017146110534668, -0.08690005540847778, 0.015777653083205223, 0.17914725840091705, -0.05087846517562866, -0.1075865775346756, 0.07873073220252991, 0.11786238104104996, -0.13607273995876312, 0.032788656651973724, -0.02365686185657978, -0.024642130360007286, -0.01521546021103859, 0.05497511103749275, 0.015562983229756355, 0.001632586237974465, -0.0018723257817327976, 0.13153427839279175, 0.05856858938932419, -0.165452241897583, 0.12432560324668884, -0.09857319295406342, 0.051370106637477875, 0.08860206604003906, 0.07020648568868637, -0.021888084709644318, -0.014020447619259357, 0.10184010863304138, -0.030455689877271652, 0.05415099486708641, -0.12555177509784698, 0.043786950409412384, -0.1507701575756073, -0.11634932458400726, -0.07047178596258163, -0.01974431425333023, -0.07292885333299637, -0.13477878272533417, -0.09503527730703354, -0.16524818539619446, -0.0854998305439949, 0.05437908321619034, 0.022116700187325478, 0.08143502473831177, 0.09557723999023438, -0.09991671144962311, -0.03244606405496597, 0.04793739691376686, -0.024332180619239807, 0.131732776761055, -0.033090803772211075, -0.02538694441318512, 0.21613939106464386, 0.08050274848937988, 0.11276966333389282, -0.04874030500650406, -0.10382478684186935, 0.002172252396121621, -0.11020981520414352, 0.1475096046924591, -0.07789572328329086, 0.059586361050605774, 0.16999858617782593, -0.00038775597931817174, -0.0764472633600235, -0.023607680574059486, 0.16016948223114014, -0.11740732938051224, -0.12456004321575165, -0.120655857026577, 0.10375886410474777, -0.010069352574646473, -0.02374737150967121, 0.01082935743033886, -0.09283221513032913, -0.045036762952804565, 0.024942094460129738, 0.01920439302921295, -0.10589013248682022, 0.011437477543950081, 0.07160980999469757, -0.05932188034057617, 0.04318370297551155, -0.10317263007164001, -0.10292788594961166, 0.0742681622505188, -0.07177155464887619, -0.02152078039944172, -0.04414068162441254, 0.0859493762254715, 0.07298517227172852, 0.07569748163223267, -0.03688504919409752, 0.06862957030534744, 0.12517784535884857, -0.030761772766709328, -0.061653416603803635, -0.07749070972204208, 0.03254144638776779, 0.018856653943657875, 0.03092668019235134, -0.020300069823861122, 0.13158996403217316, -0.16806909441947937, -0.14242233335971832, 0.010574812069535255, -0.025519447401165962, -0.005407075863331556, 0.10722548514604568, -0.04233415052294731, 0.08546183258295059, 0.1798471212387085, 0.10849184542894363, 0.11706861853599548, 0.06605040282011032, 0.11442794650793076, -0.021510709077119827, 0.09933127462863922, -0.02331787347793579, -0.020526569336652756, -0.07022701948881149, -0.025708917528390884, -0.15789495408535004, 0.038784872740507126, 0.048561736941337585, 0.08245661854743958, -0.0034329346381127834, 0.03724195063114166, 0.12646609544754028, -0.050205543637275696, 0.12626908719539642, -0.09568551927804947, 0.05418061092495918, 0.02703128196299076, -0.11990436911582947, -0.026327023282647133, -0.014542528428137302, 0.10807988792657852, 0.06854884326457977, 0.06423301249742508, -0.10356446355581284, 0.015992172062397003, -0.03863650932908058, 0.0025141441728919744, 0.0873635783791542, 0.03513193503022194, -0.06632372736930847, -0.010131229646503925, -0.1271577924489975, 0.027227025479078293, 0.18051216006278992, -0.004837338346987963, -0.06885575503110886, -0.016189292073249817, -0.0345696322619915, 0.057895779609680176, 0.1008201315999031, -0.17738305032253265, -0.044114381074905396, 0.09283699095249176, 0.031277429312467575, -0.11169835925102234, 0.06068969517946243, 0.1316826343536377, -0.020918548107147217, -0.06070882827043533, -0.038153812289237976, -0.08577083051204681, 0.04276103526353836, 0.12004091590642929, -0.009270239621400833, 0.10349492728710175, 0.030817773193120956, -0.02461167797446251, -0.0531708225607872, 0.003025529906153679, -0.02718402072787285, -0.0379483625292778, 0.005360641051083803, -0.09115282446146011, 0.010739720426499844, -0.03203573822975159, -0.005037171766161919, -0.058126792311668396, 0.10723157972097397, -0.02752356231212616, 0.031089400872588158, -0.002061927691102028, -0.07504651695489883, -0.058851249516010284, -0.13772669434547424, -0.11866407096385956, 0.04874967411160469, 0.044476401060819626, 0.1112649217247963, 0.07714400440454483, 0.11957471072673798, 0.09043631702661514, 0.12484342604875565, 0.0535527803003788, -0.07818603515625, -0.03417910262942314, 0.06233901530504227, -0.06857237964868546, 0.09282869845628738, 0.0014028449077159166, -0.08076856285333633, 0.0036932192742824554, 0.0033810012973845005, -0.006729753687977791, 0.006712656002491713, -0.04664798080921173, -0.05427386984229088, 0.0534786619246006, 0.01897016540169716, -0.05647839233279228, -0.03843053802847862, -0.03997902199625969, -0.06514451652765274, 0.11994954943656921, -0.025672148913145065, -0.1062687486410141, 0.04327382519841194, 0.021263254806399345, 0.06780967116355896, 0.001441184664145112, 0.08680207282304764, 0.014493177644908428, 0.12759098410606384, -0.08776439726352692, 0.13423505425453186, -0.03635195270180702, 0.02862633764743805, 0.04621797800064087, 0.02696629986166954, 0.06947030127048492, 0.1256161779165268, 0.05799451097846031, -0.13739635050296783, -0.027424195781350136, 0.05576414242386818, 0.003243883140385151, 0.00773437088355422, -0.08844486624002457, 0.07394286245107651, -0.11889168620109558, -0.085075743496418, 0.07702157646417618, -0.05303855240345001, 0.010861179791390896, 0.05568915605545044, 0.09362595528364182, -0.08017898350954056, 0.04426611214876175, 0.017229149118065834, -0.09203609824180603, 0.10661322623491287, -0.011377674527466297, 0.035533688962459564, -0.11589016765356064, -0.12737932801246643, -0.12614154815673828, -0.0034486509393900633, -0.01078629121184349, 0.04202652350068092, 0.06538186967372894, -0.085311159491539, -0.0643593892455101, 0.04777887463569641, -0.05342179164290428, -0.056576721370220184, -0.002014331752434373, -0.0815463438630104, 0.07406352460384369, 0.055164311081171036, -0.02242393046617508, -0.17491944134235382, 0.012617210857570171, -0.13510169088840485, -0.1266094297170639, 0.11108618229627609, -0.06396239250898361, 0.1302490532398224, -0.05843663215637207, -0.07555817812681198, 0.044193562120199203, -0.02208716608583927, -0.05248617008328438, 0.08079537004232407, -0.017185255885124207, 0.049533482640981674, 0.07785528898239136, 0.025510670617222786, -0.12506499886512756, 0.06272199004888535, -0.022785115987062454, -0.08441445976495743, 0.12396474182605743, -0.051887087523937225, -0.0433666817843914, -0.09434522688388824, 0.008116990327835083, 0.07324568182229996, -0.08206453919410706, -0.00012768810847774148, 0.015604672953486443, 0.02705490216612816, -0.11831264942884445, 0.0628010556101799, -0.0733245313167572, -0.030497604981064796, -0.04183679446578026, -0.033295486122369766, -0.13131354749202728, -0.055480264127254486, 0.10410572588443756, 0.11002333462238312, -0.0013353157555684447, -0.05313475430011749, 0.07208525389432907, 0.07858514785766602, 0.012871244922280312, 0.0046269865706563, -0.07454852759838104, -0.0842619314789772, -0.05128791183233261, 0.05305512621998787, 0.08876553922891617, 0.06790820509195328, -0.0052104610949754715, -0.025298764929175377, -0.03510674089193344, -0.045989684760570526, 0.03337092325091362, -0.12280960381031036, 0.037814199924468994, 0.0896497592329979, -0.09589822590351105, -0.09947242587804794, -0.05686161294579506, 0.01619686372578144, -0.008502229116857052, 0.12166649848222733, -0.09855448454618454, 0.0369514599442482, 0.09670227020978928, 0.09942387789487839, -0.011339024640619755, 0.04393228515982628, -0.08962985128164291, 0.03287302330136299, -0.04895948991179466, 0.08620563894510269, -0.07615730166435242, -0.006566155236214399, -0.022125044837594032, -0.004259579814970493, 0.013117671012878418, 0.011482872068881989, -0.16601286828517914, 0.08126363158226013, 0.021137770265340805, -0.04086041450500488, -0.02406812645494938, -0.10381234437227249, 0.0183408185839653, -0.032011453062295914, -0.04900430142879486, 0.049226101487874985, 0.0740058645606041, 0.022649919614195824, 0.08227506279945374, 0.03150831535458565, 0.11179686337709427, 0.011017778888344765, -0.06712966412305832, -0.04982585459947586, 0.014727356843650341, -0.07639370858669281, -0.1048600897192955, -0.010030343197286129, -0.13312968611717224, 0.05631760507822037, -0.031192172318696976, -0.09144799411296844, 0.03875596448779106, 0.00900951400399208, 0.09990114718675613, -0.00770106865093112, 0.13562199473381042, 0.053572364151477814, -0.030947156250476837, 0.13000887632369995, -0.038896311074495316, 0.06626404076814651, 0.0055656470358371735, 0.0022343446034938097, -0.017291737720370293, 0.03642788156867027, 0.04642913118004799, 0.06079325079917908, 0.031450510025024414, 0.008404364809393883, 0.15712110698223114, -0.05497468262910843, 0.045248255133628845, -0.03663618117570877, -0.08232655376195908, 0.026757745072245598, -0.003896944923326373, 0.09131388366222382, 0.04951677843928337, 0.013595580123364925, -0.10839910060167313, 0.13719172775745392, -0.0246263537555933, 0.037557899951934814, -0.004152148962020874, 0.015909574925899506, 0.08141466975212097, 0.06009209528565407, 0.07177890837192535, 0.019740432500839233, 0.030354401096701622, 0.07533956319093704, 0.02997889742255211, -0.0052694277837872505, -0.035908281803131104, -0.11638034880161285, 0.07082447409629822, -0.10045648366212845, 0.0962960422039032, -0.03022794984281063, -0.08778169751167297, -0.014809232205152512, 0.011354918591678143, -0.11988189816474915, -0.05630560591816902, -0.0993766263127327, 0.0048504481092095375, -0.038672856986522675, 0.03985455259680748, -0.08177953958511353, 0.1350543200969696, 0.1094442754983902, 0.08788321167230606, 0.01737748086452484, -0.1354716718196869, 0.03263681381940842, -0.03569147735834122, -0.06567142903804779, 0.04853415489196777, 0.14321865141391754, 0.014675015583634377, -0.11773771792650223, 0.08878397196531296, 0.010805957019329071, -0.01334813516587019, -0.008671721443533897, -0.028085386380553246, 0.04878616705536842, 0.026838218793272972, 0.11564347892999649, 0.07688819617033005, -0.10028109699487686, -0.06318392604589462, -0.10154886543750763, -0.07484525442123413, -0.02994072623550892, -0.04707343131303787, 0.05907145142555237, -0.06549333781003952, 0.04307842254638672, -0.052595775574445724, -0.09579230099916458, -0.02848808839917183, 0.00787059124559164, -0.17221641540527344, 0.1452871412038803, -0.0816296935081482, -0.03809933364391327, 0.03856848552823067, -0.07159459590911865, -0.027433158829808235, 0.02667061984539032, 0.031059863045811653, -0.017488358542323112, -0.09696683287620544, -0.0635438933968544, 0.06330110132694244, -0.002101112389937043, -0.07043453305959702, -0.08662579208612442, 0.08191506564617157, -0.0953483060002327, 0.04698380455374718, -0.04473412409424782, -0.06676781177520752, 0.035015400499105453, 0.017414068803191185, -0.11535648256540298, 0.008555077016353607, 0.016141684725880623, 0.030392128974199295, 0.1271953582763672, -0.142067089676857, -0.11446675658226013, -0.12884104251861572, -0.01654387265443802, 0.080927774310112, 0.03164505958557129, -0.06954864412546158, 0.029637344181537628, 0.009819906204938889, -0.05285251513123512, -0.05036300793290138, 0.006058323197066784, 0.09782446920871735, 0.0922776535153389, -0.09341910481452942, -0.12276934832334518, -0.0688694640994072, 0.008097310550510883, -0.0012262947857379913, -0.053228702396154404, 0.0815664678812027, -0.07181031256914139, -0.053546927869319916, -0.012068443931639194, -0.014370230957865715, 0.008652358315885067, -0.07545033097267151, 0.09789874404668808, -0.011243147775530815, -0.0188487209379673, 0.1238776370882988, 0.058345481753349304, -0.022289065644145012, -0.0841212347149849, 0.010178076103329659, 0.034869056195020676, 0.03965090215206146, -0.06029432639479637, 0.08406050503253937, -0.08086193352937698, -0.10599301755428314, -0.01885348930954933, 0.11769974231719971, -0.0792074054479599, 0.1690714955329895, -0.04427282139658928, 0.032575033605098724, 0.09902498871088028, -0.03820925951004028, -0.028146961703896523, -0.051016561686992645, 0.001948143937624991, 0.10841232538223267, -0.018976489081978798, -0.04197671636939049, 0.06238774582743645, 0.10730299353599548, -0.07389257848262787, -0.01951034925878048, -0.03873588144779205, -0.09416741132736206, -0.10495539754629135, -0.021695807576179504, -0.07667127996683121, 0.0831478089094162, 0.08604291826486588, 0.0769883468747139, 0.04998279735445976, 0.1761789172887802, 0.062489449977874756, -0.09033714234828949, 0.12195087969303131, -0.10787678509950638, 0.039820071309804916, -0.0374944768846035, 0.052411530166864395, -0.1477273553609848, 0.06197882816195488, 0.11491794884204865, 0.10466104745864868, -0.16205385327339172, -0.030268143862485886, -0.062050413340330124, -0.09961052983999252, -0.19145700335502625, 0.08681339770555496, -0.14845146238803864, 0.014007799327373505, 0.12644332647323608, -0.10725649446249008, -0.029636360704898834, -0.010809454135596752, -0.046893972903490067, 0.007930781692266464, 0.07521745562553406, 0.007883292622864246, -0.11141195893287659, -0.09153731167316437, -0.01228674128651619, 0.04585961252450943, 0.02361210808157921, 0.02880595065653324, -0.027807585895061493, 0.13149592280387878, -0.03999593108892441, -0.00028037195443175733, -0.10591819137334824, 0.09577571600675583, -0.06016763672232628, 0.08852548897266388, -0.018919264897704124, -0.04472081735730171, -0.08838503807783127, 0.05741661787033081, -0.12710028886795044, -0.058544259518384933, 0.08107538521289825, -0.002539954148232937, 0.0036477011162787676, 0.057956986129283905, -0.14561717212200165, -4.328911018092185e-05, 0.05046146363019943, 0.0944468155503273, -0.02330867014825344, -0.14585226774215698, -0.1501299887895584, -0.053210802376270294, 0.04647087678313255, -0.08535529673099518, -0.08595570176839828, -0.11342980712652206, -0.017455577850341797, 0.07846450805664062, 0.10178820788860321, -0.06083133816719055, 0.06292186677455902, 0.09714240580797195, 0.06362126767635345, -0.04197372868657112, 0.02350744418799877, -0.058226827532052994, 0.07166768610477448, -0.015536008402705193, 0.014064615592360497, -0.011438794434070587, 0.08078283816576004, 0.07544687390327454, 0.015938159078359604, -0.04240129515528679, 0.015364975668489933, 0.06387580931186676, 0.0769789069890976, 0.04939427226781845, -0.04968959838151932, 0.05806247144937515, -0.16593198478221893, -0.035537946969270706, -0.07797590643167496, -0.0133884372189641, -0.12281394004821777, 0.0059012859128415585, -0.08074527978897095, 0.1496928334236145, 0.038456693291664124, -0.08940330147743225, -0.010828087106347084, 0.013032936491072178, 0.025678232312202454, 0.06513328850269318, -0.03838036209344864, -0.008308837190270424, -0.04369828477501869, 0.06744731962680817, 0.06986343860626221, 0.055221524089574814, -0.05010479688644409, 0.08294513821601868, 0.1050465777516365, -0.08606579154729843, 0.12120155245065689, -0.14548087120056152, -0.11301281303167343, -0.10757284611463547, -0.009345095604658127, -0.02866615727543831, 0.13874149322509766, -0.04824360832571983, 0.044832680374383926, -0.13546150922775269, -0.06010940670967102, -0.045191340148448944, 0.004681560676544905, -0.01802392676472664, -0.05285695940256119, -0.12644818425178528, 0.08184847235679626, 0.05672182887792587, -0.04757582023739815, 0.1772449016571045, -0.018052905797958374, 0.16130904853343964, -0.030985545367002487, 0.11324625462293625, -0.023533478379249573, -0.0331999734044075, 0.03304589167237282, -0.04526733234524727, -0.03835776820778847, -0.0916600301861763, 0.08435703814029694, -0.03832855075597763, 0.03586701303720474, -0.07781950384378433, -0.07720765471458435, -0.09951620548963547, -0.05730602517724037, 0.01248251460492611, -0.07067517191171646, -0.14269745349884033, -0.05086681619286537, 0.018561722710728645, -0.11217064410448074, -0.02273915708065033, -0.0164300836622715, -0.08818073570728302, -0.009816613979637623, 0.006676956079900265, -0.1326339691877365, 0.07352801412343979, -0.10629507154226303, -0.013602755032479763, -0.08860581368207932, -0.12926983833312988, -0.07706459611654282, -0.07519805431365967, 0.08166678249835968, 0.05462648347020149, 0.08545638620853424, 0.09137888997793198, -0.020332084968686104, 0.06535183638334274, -0.0860062763094902, -0.06858750432729721, 0.0992029532790184, 0.11857258528470993, 0.06638998538255692, -0.052143022418022156, -0.16382436454296112, -0.11462648957967758, 0.09981511533260345, 0.07835903018712997, -0.03651779145002365, 0.027308663353323936, 0.018645890057086945, -0.11426422744989395, -0.14458228647708893, -0.15674324333667755, -0.14922626316547394, -0.0645948201417923, 0.14007902145385742, -0.0028516906313598156, -0.02376118302345276, -0.04389052465558052, -0.07755373418331146, 0.0775919109582901, 0.10328033566474915, 2.522798786230851e-05, 0.14998304843902588, 0.09906041622161865, -0.07479413598775864, 0.11057238280773163, -0.02757486142218113, -0.13326960802078247, -0.005966945085674524, 0.0692724883556366, -0.04148874431848526, 0.1329573392868042, -0.07465063780546188, 0.08220496773719788, 0.05633743107318878, 0.010559325106441975, -0.08404373377561569, 0.012855506502091885, 0.08019908517599106, 0.14963923394680023, -0.08789361268281937, -0.05129861831665039, -0.05850633233785629, -0.13665394484996796, 0.06962786614894867, -0.1036251112818718, 0.14044155180454254, 0.031463321298360825, -0.06919468939304352, 0.09725701808929443, 0.16478367149829865, 0.03802280128002167, -0.08736018091440201, -0.013961232267320156, -0.031001852825284004, 0.04086815193295479, -0.10399406403303146, 0.06127959117293358, -0.11955302953720093, -0.060457583516836166, 0.15288902819156647, 0.17434099316596985, 0.027016237378120422, 0.008094887249171734, 0.02860303223133087, 0.08323314040899277, -0.08006861805915833, -0.03346486762166023, 0.07925014197826385, 0.036790162324905396, 0.11230623722076416, 0.05938086658716202, -0.03199339285492897, 0.012763225473463535, -0.008499311283230782, 0.17183583974838257, -0.030098287388682365, -0.10784859955310822, -0.08979908376932144, -0.11971662938594818, -0.09794659912586212, 0.005301911383867264, -0.06757382303476334, -0.05734230950474739, 0.03600111976265907, -0.03409259393811226, 0.014535805210471153, 0.0332186222076416, 0.10665643215179443, -0.10859227180480957, -0.005183515604585409, -0.01150986086577177, 0.05764997750520706, -0.023227110505104065, -0.039564087986946106, 0.07111304253339767, 0.0009707969147711992, 0.03173331543803215, -0.0903245359659195, -0.06782835721969604, 0.10043615847826004, 0.05979468300938606, 0.09257740527391434, -0.013017138466238976, 0.04428324103355408, -0.06157897412776947, -0.09055228531360626, -0.030977947637438774, 0.13623322546482086, 0.09951992332935333, 0.03097166307270527, 0.029040632769465446, 0.10993992537260056, 0.02772899530827999, -0.0978512093424797, -0.11468438804149628, 0.11595027148723602, -0.053273897618055344, -0.05293017625808716, 0.058548104017972946, 0.0022509319242089987, 0.004305866081267595, 0.06215616688132286, 0.10606447607278824, 0.02477790042757988, 0.044031694531440735, 0.06513936072587967, -0.044979702681303024, -0.09485378861427307, -0.07982935011386871, -0.15587005019187927, 0.018895557150244713, -0.12029267102479935, 0.161367729306221, 0.0513286292552948, -0.10267540067434311, 0.0785740315914154, -0.016294829547405243, -0.05107005313038826, -0.11803138256072998, -0.04789802059531212, -0.01980942115187645, 0.16620072722434998, 0.03225605562329292, -0.05484098941087723, -0.011866169981658459, -0.038502972573041916, -0.03344578668475151, 0.007642888464033604, 0.09469208121299744, 0.16174183785915375, -0.08251107484102249, -0.03554638475179672, -0.04956014081835747, 0.09548301249742508, 0.09553982317447662, -0.0168879684060812, 0.12773507833480835, -0.08177439868450165, 0.002322036772966385, -0.001467136899009347, 0.11403146386146545, 0.14534509181976318, -0.044425711035728455, 0.08782534301280975, -0.07837355136871338, -0.0011491387849673629, -0.08259258419275284, 0.13066628575325012, -0.07844842970371246, -0.10346834361553192, 0.04270920529961586, -0.04247397929430008, 0.04928034171462059, -0.09694524854421616, -0.041024744510650635, -0.06691519170999527, 0.08065026998519897, 0.10749293863773346, -0.012807967141270638, 0.05486384406685829, -0.02819492667913437, 0.044636692851781845, -0.015613841824233532, 0.02399141900241375, 0.0921994298696518, 0.08249709755182266, -0.15173572301864624, -0.03575080633163452, -0.03830987215042114, -0.05458815395832062, -0.08268525451421738, -0.013665968552231789, -0.035844817757606506, -0.10338451713323593, 0.1290101706981659, 0.048264119774103165, -0.12062583118677139, -0.10630029439926147, -0.0641573816537857, -0.07381149381399155, -0.029144395142793655, -0.08357057720422745, 0.1188102439045906, 0.026689616963267326, 0.05255233868956566, 0.03488565981388092, -0.09069134294986725, 0.1089114174246788, -0.07375217229127884, 0.06131843477487564, 0.000374789466150105, 0.07075335085391998, 0.0659472644329071, 0.052324675023555756, 0.03899797052145004, 0.08832952380180359, 0.05676504969596863, 0.029784057289361954, 0.044098690152168274, 0.12290121614933014, 0.10083655267953873, 0.03807821497321129, -0.06696310639381409, -0.0762462317943573, 0.10911335051059723, 0.012042212300002575, -0.02508651278913021, 0.0666976198554039, 0.0020471508614718914, 0.01264561153948307, -0.024546939879655838, -0.005996434483677149, -0.09605366736650467, 0.12535151839256287, 0.0690004900097847, -0.024243075400590897, 0.029366860166192055, -0.05425266921520233, 0.13906092941761017, -0.10552272200584412, -0.08045703917741776, 0.043590396642684937, -0.02130405232310295, 0.05881805345416069, 0.05911974981427193, 0.057589393109083176, 0.11212438344955444, 0.0729430541396141, 0.015544474124908447, -0.03702468052506447, -0.10526937246322632, -0.008547944016754627, 0.05797486752271652, -0.08261216431856155, -0.0006268128636293113, 0.08364711701869965, 0.04429091513156891, -0.000603016116656363, -0.008687385357916355, 0.05057525634765625, -0.1272706836462021, 0.09562128037214279, 0.00019482587231323123, 0.033198267221450806, 0.18214097619056702, 0.053530093282461166, 0.10066637396812439, 0.12424222379922867, 0.06416704505681992, -0.15193752944469452, 0.02306538075208664, -0.0680689886212349, -0.15204113721847534, 0.028588689863681793, 0.056385744363069534, 0.08225171267986298, 0.097345732152462, 0.11498920619487762, 0.04321646690368652, -0.06122790649533272, 0.054301079362630844, 0.04788721352815628, 0.10711321234703064, 0.005414060782641172, 0.06874151527881622, 0.05951342359185219, 0.09852944314479828, 0.0090187294408679, 0.1046091765165329, 0.10009056329727173, -0.05707746744155884, -0.06276487559080124, -0.06950273364782333, -0.05455264076590538, -0.10702855885028839, -0.043883051723241806, -0.16808107495307922, 0.08338315039873123, 0.13599011301994324, 0.01479326095432043, -0.03233739733695984, -0.15012980997562408, -0.13938981294631958, -0.13122837245464325, -0.007913950830698013, -0.10149089246988297, 0.08781643956899643, 0.02240082435309887, 0.11388592422008514, -0.14202812314033508, 0.04072083532810211, 0.038062140345573425, -0.08142639696598053, 0.09665309637784958, -0.049200624227523804, 0.07998483628034592, -0.15359419584274292, 0.04486601799726486, -0.06430625170469284, 0.014434105716645718, -0.08323729783296585, -0.01938350684940815, -0.10597550123929977, -0.011050821281969547, -0.039072100073099136, 0.017543034628033638, 0.01135488972067833, -0.1676081120967865, 0.03061455301940441, 0.000273371726507321, 0.07354608178138733, -0.06864949315786362, -0.004144623409956694, -0.02860206365585327, 0.0596870481967926, -0.01626243256032467, 0.0763566642999649, 0.09192462265491486, -0.09342663735151291, 0.011220457032322884, -0.023112433031201363, 0.04728955402970314, 0.10756991058588028, -0.15413795411586761, -0.01865379698574543, 0.06339883804321289, -0.030139757320284843, 0.02157525345683098, 0.00759280426427722, 0.030539147555828094, -0.12264163047075272, -0.10960591584444046, -0.0015171042177826166, -0.0072916722856462, 0.05550214648246765, -0.0029010921716690063, -0.03629446402192116, 0.05770295113325119, -0.035326793789863586, -0.009376415982842445, -0.13510417938232422, 0.11411937326192856, -0.061107344925403595, 0.03081684187054634, 0.040873173624277115, -0.0972183495759964, 0.08090461045503616, 0.012514953501522541, 0.08998887240886688, -0.008400117047131062, 0.06705565005540848, 0.0590435154736042, 0.0520002618432045, -0.029151806607842445, -0.006446978077292442, -0.00897151418030262, 0.0860292986035347, 0.10006083548069, -0.10477907210588455, -0.006394879426807165, 0.07502627372741699, 0.025476114824414253, 0.10170479118824005, 0.1368888020515442, -0.004469391889870167, 0.1014738604426384, 0.025778932496905327, -0.06268694251775742, 0.02085631899535656, -0.08252064883708954, -0.08313734084367752, -0.018516136333346367, -0.05257757753133774, 0.12221196293830872, 0.04540213569998741, -0.12037289142608643, 0.07384684681892395, -0.0014714639401063323, 0.03620528429746628, 0.033213499933481216, 0.08105399459600449, -0.07214199006557465, -0.09987717866897583, 0.043228138238191605, 0.10026411712169647, 0.09854812175035477, 0.07956628501415253, -0.05589699000120163, 0.030147382989525795, 0.0034314219374209642, -0.05050981044769287, -0.0786234587430954, -0.056237153708934784, -0.08675315231084824, 0.10004692524671555, 0.07599566131830215, -0.026213867589831352, 0.056183505803346634, -0.08497986197471619, -0.003486095927655697, 0.029412774369120598, 0.04867541790008545, -0.07793410867452621, 0.016019806265830994, 0.06728041917085648, 0.039440128952264786, -0.07464151829481125, -0.04524625837802887, 0.10045354068279266, 0.025857161730527878, -0.01847471110522747, 0.04159196466207504, 0.007829112932085991, 0.061656270176172256, 0.02846837416291237, -0.10448125004768372, 0.018759580329060555, 0.02919280342757702, 0.01042647659778595, 0.10694397240877151, -0.02434580959379673, 0.1141931340098381, -0.08974502235651016, -0.1369745135307312, 0.1430179625749588, -0.09063636511564255, -0.014197343029081821, -0.08017493784427643, -0.02406003512442112, 0.011554530821740627, 0.05757175758481026, -0.03434018790721893, -0.023547455668449402, 0.0433645099401474, -0.04555116966366768, 0.1401648223400116, -0.045509111136198044, 0.06836574524641037, 0.02105838619172573, 0.05766340345144272, 0.07184415310621262, -0.003398194210603833, 0.10071280598640442, 0.026883671060204506, -0.09247750043869019, -0.044576458632946014, -0.03140382468700409, -0.05495723336935043, -0.1348290890455246, -0.0058273449540138245, -0.03647206723690033, -0.10327237099409103, -0.06278710067272186, 0.08764124661684036, -0.03809362277388573, 0.02919607423245907, 0.015276886522769928, -0.009784308262169361, 0.1283966451883316, 0.13239842653274536, 0.105104461312294, 0.010916478000581264, 0.11822988092899323, -0.12161751091480255, -0.030621690675616264, 0.0045273443683981895, 0.11652325093746185, 0.04973189905285835, 0.01937120594084263, -0.06251920759677887, -0.05901692062616348, 0.07250114530324936, 0.020844368264079094, -0.005729851312935352, -0.12461232393980026, 0.07335902750492096, 0.026987383142113686, 0.11905395984649658, -0.07225378602743149, -0.03498835116624832, -0.000931376707740128, 0.039099082350730896, -0.02471189759671688, -0.05601559951901436, -0.09262949973344803, 0.016723565757274628, -0.026244184002280235, 0.007174327969551086, -0.14521242678165436, -0.07218510657548904, -0.06141234561800957, 0.03385080769658089, -0.043001480400562286, -0.07570665329694748, -0.006005527917295694, 0.03221728280186653, -0.08852510899305344, -0.06744027137756348, -0.03652724251151085, -0.0727662742137909, -0.08283542841672897, 0.13884848356246948, 0.1293870210647583, -0.06130358204245567, 0.0325317308306694, 0.00673661520704627, 0.1392170935869217, 0.07268137484788895, 0.014998898841440678, -0.05926956608891487, 0.013591563329100609, -0.11739885061979294, -0.02336941845715046, -0.14977560937404633, -0.021574702113866806, 0.012338714674115181, -0.15016460418701172, 0.03348954766988754, -0.08658774197101593, 0.08952014148235321, 0.06582087278366089, 0.14301781356334686, -0.0568259097635746, -0.0810614600777626, -0.012931520119309425, 0.04082825034856796, -0.06981207430362701, 0.07565972954034805, 0.10654119402170181, -0.04298070818185806, -0.03023015707731247, -0.0662362203001976, 0.16230492293834686, -0.017761733382940292, 0.02662617154419422, 0.08281668275594711, 0.0777045413851738, 0.04474910348653793, 0.007188357412815094, 0.06002087891101837, -0.07075009495019913, -0.13255169987678528, 0.010449440218508244, -0.014012814499437809, -0.12010857462882996, -0.020968014374375343, -0.08964154124259949, 0.005155892111361027, 0.004116280470043421, -0.01737573929131031, -0.07757845520973206, -0.12342389672994614, -0.13345856964588165, -0.06147919222712517, 0.10470247268676758, 0.006270568817853928, -0.12871165573596954, -0.05136297270655632, -0.03027649223804474, 0.1064891368150711, -0.01209137961268425, 0.034841425716876984, -0.039200421422719955, -0.05598665401339531, -0.10422692447900772, 0.02930130623281002, 0.04014192149043083, -0.010138062760233879, 0.019881024956703186, -0.14801034331321716, 0.0712098628282547, -0.09984719008207321, -0.06223692744970322, -0.039879210293293, 0.06335911154747009, 0.12947183847427368, -0.02625110000371933, 0.1290389597415924, 0.004587747156620026, 0.05222172662615776, -0.06081079691648483, -0.003392778569832444, -0.036541543900966644, 0.03717942535877228, 0.019615748897194862, -0.10087156295776367, 0.05562397092580795, 0.017716383561491966, -0.08010726422071457, 0.12001197785139084, 0.028199756518006325, -0.08769648522138596, -0.029192069545388222, -0.12183648347854614, 0.0799187421798706, 0.11514972895383835, 0.03880475088953972, -0.06949427723884583, 0.08409962058067322, 0.0379183366894722, -0.11286509037017822, -0.10213194787502289, 0.02062898874282837, -0.12682433426380157, -0.02457735314965248, -0.04234253242611885, -0.03875201568007469, 0.08492086827754974, -0.09851688891649246, 0.14304769039154053, -0.022602969780564308, 0.03312045335769653, -0.048254407942295074, -0.01225457713007927, -0.0160091370344162, -0.09037067741155624, -0.05045657232403755, 0.11981826275587082, 0.014157863333821297, 0.10870686918497086, -0.11462101340293884, 0.021848615258932114, -0.027436470612883568, 0.011136524379253387, -0.11003918945789337, 0.11501895636320114, -0.0409478098154068, -0.05348170921206474, -0.07648047804832458, -0.01985717937350273, -0.062251653522253036, -0.12084852904081345, 0.13815169036388397, -0.06926377862691879, -0.029882144182920456, -0.11442405730485916, -0.054515425115823746, 0.06165149062871933, 0.11070813238620758, 0.052567820996046066, 0.05051906034350395, 0.10822918266057968, 0.04223442077636719, 0.05645018815994263, 0.03106960654258728, 0.08008622378110886, 0.0753755271434784, 0.10687121003866196, 0.015681030228734016, -0.009469395503401756, -0.11163207143545151, 0.1122535690665245, 0.04270496591925621, 0.09655170887708664, 0.006961137522011995, -0.057212941348552704, 0.0021031382493674755, -0.11124470829963684, -0.06078261137008667, 0.05795520544052124, 0.026543013751506805, 0.11104905605316162, -0.02138740010559559, -0.09094999730587006, -0.041172053664922714, -0.0213327556848526, -0.021514443680644035, -0.1240006610751152, 0.09431200474500656, 0.08587003499269485, 0.08140899240970612, -0.005258568562567234, 0.03609587997198105, -0.09087828546762466, -0.044759601354599, -0.023630429059267044, -0.06472063809633255, -0.07857055962085724, -0.011368941515684128, 0.013396174646914005, -0.06075504049658775, -0.06898103654384613, 0.0561983548104763, 0.040861696004867554, -0.08426462858915329, 0.09293396770954132, 0.005982176400721073, -0.09968419373035431, -0.048612769693136215, -0.040351517498493195, -0.10187402367591858, 0.018635842949151993, -0.007454392965883017, -0.011356577277183533, -0.0006113645504228771, -0.07720474898815155, -0.11323180794715881, -0.020396962761878967, -0.017357930541038513, 0.09386321157217026, 0.09700312465429306, 0.07259494066238403, -0.017747165635228157, -0.014437645673751831, 0.12288779765367508, 0.11111370474100113, 0.029857268556952477, 0.08046139031648636, 0.06352423876523972, -0.06773210316896439, -0.03109397180378437, 0.005041773896664381, 0.058819256722927094, 0.0333358570933342, -0.0487929992377758, -0.056026048958301544, -0.1373005509376526, -0.08332347124814987, 0.013893584720790386, -0.06170496344566345, 0.05612977221608162, -0.04553349316120148, -0.10771038383245468, -0.004512601997703314, 0.07527922838926315, 0.033354274928569794, 0.07689778506755829, -0.08094321936368942, -0.0644955262541771, -0.0722503736615181, -0.036635566502809525, 0.07545315474271774, 0.08467543870210648, 0.02048759162425995, -0.07885298132896423, -0.040842454880476, 0.0675729438662529, -0.03591875359416008, 0.0867551788687706, 0.0628046914935112, 0.05846741423010826, -0.06583832204341888, 0.06465496867895126, -0.022622408345341682, 0.027365101501345634, -0.019116586074233055, -0.0039662872441112995, -0.09031100571155548, 0.02059105969965458, -0.11067314445972443, 0.13277357816696167, 0.004604698624461889, -0.045920584350824356, 0.010743041522800922, 0.10545073449611664, -0.047639425843954086, -0.06716234982013702, 0.03981879726052284, -0.0639166533946991, -0.0958251804113388, -0.03506934270262718, 0.0877293199300766, -0.03975534439086914, -0.09434478729963303, -0.0453290119767189, -0.0608203299343586, 0.05304248631000519, 0.007914441637694836, -0.07815077155828476, 0.031611520797014236, -0.12864871323108673, -0.12608502805233002, -0.06556351482868195, 0.06666935235261917, 0.07019846141338348, 0.0071296775713562965, 0.0038574659265577793, -0.011345122940838337, 0.07486240565776825, -0.03804875165224075, 0.0910392478108406, -0.06395133584737778, -0.07411397248506546, -0.019009580835700035, -0.1137814149260521, 0.023836884647607803, -0.0634622648358345, 0.04143604263663292, -0.07221879065036774, -0.05414589121937752, -0.04240012168884277, 0.12952972948551178, -0.015807025134563446, 0.024623222649097443, 0.1002495214343071, -0.05931652709841728, -0.06289640069007874, -0.10859301686286926, -0.008288128301501274, -0.09595775604248047, -0.025164533406496048, -0.11829342693090439, 0.03327400982379913, -0.03228919580578804, 0.02493174560368061, 0.05295109003782272, -0.014153724536299706, -0.040981538593769073, 0.004809585865586996, 0.06663013994693756, 0.07073750346899033, -0.09874821454286575, 0.0976443886756897, 0.025043394416570663, 0.0038560954853892326, 0.01568072848021984, -0.0041824812069535255, 0.07295636832714081, -0.09059876203536987, -0.04939483851194382, 0.11695100367069244, -0.06289321929216385, -0.00036784744588658214, 0.07447865605354309, -0.045577649027109146, 0.0752323567867279, 0.013172021135687828, -0.012814059853553772, 0.09039275348186493, -0.04637879133224487, -0.0008147581247612834, -0.06844436377286911, -0.016255831345915794, 0.08768423646688461, 0.020435435697436333, -0.03903254494071007, -0.09229466319084167, 0.06926189363002777, 0.04973975196480751, -0.11004690825939178, -0.005055315792560577, -0.09275442361831665, 0.014085001312196255, -0.02898891270160675, -0.1232355609536171, -0.07918603718280792, 0.07918436825275421, 0.04671649634838104, -0.0077177113853394985, 0.07497666031122208, 0.015343242324888706, -0.08749493211507797, -0.03707784414291382, -0.11484278738498688, -0.0021865307353436947, 0.09651318192481995, 0.034061040729284286, -0.001675087260082364, 0.07535651326179504, -0.0685628354549408, -0.13009409606456757, 0.0038867443799972534, -0.14237821102142334, -0.06629816442728043, -0.10721123218536377, -0.01387597993016243, 0.05607546120882034, -0.06068119406700134, -0.028291864320635796, -0.03253396600484848, 0.11417810618877411, 0.07765573263168335, -0.03121710941195488, -6.271107122302055e-05, -0.0065526836551725864, -0.027470478788018227, 0.06802905350923538, -0.06567002087831497, 0.04814187437295914, -0.06783122569322586, -0.1408437043428421, -0.06629592925310135, 0.10058260709047318, 0.13980168104171753, -0.04741403087973595, -0.10028301924467087, -0.04965905845165253, -0.025011811405420303, -0.05313238129019737, 0.09471789002418518, 0.09635771811008453, -0.1271764487028122, 0.10346999764442444, 0.04983276501297951, -0.05450925603508949, 0.13153192400932312, -0.0696837455034256, -0.05485019460320473, -0.04784613475203514, -0.015710605308413506, 0.03537087142467499, -0.029560402035713196, -0.016263550147414207, -0.07732120156288147, 0.045968227088451385, -0.08204077929258347, -0.006486519239842892, -0.11965769529342651, -0.015519928187131882, -0.04966290295124054, 0.014805976301431656, 0.13261812925338745, 0.09314630180597305, -0.06792132556438446, 0.15366202592849731, -0.07078170031309128, 0.012056318111717701, 0.1258431375026703, 0.10239428281784058, -0.016318008303642273, -0.03689064830541611, 0.10132396966218948, 0.08246194571256638, 0.017704488709568977, -0.030136989429593086, 0.005223000422120094, -0.028481218963861465, -0.033502236008644104, 0.04479255899786949, -0.050623536109924316, -0.10773028433322906, -0.09054829180240631, 0.0777178481221199, 0.12177349627017975, 0.08872140944004059, -0.04463934525847435, 0.04385014623403549, 0.11769848316907883, -0.0420440211892128, 0.02492612414062023, 0.11256931722164154, 0.05427198484539986, -0.03482087329030037, -0.11453419178724289, -0.09092473238706589, 0.060130178928375244, 0.02862979844212532, -0.01229160651564598, 0.05204182118177414, 0.09055348485708237, -0.0048883454874157906, 0.019532086327672005, 0.09563002735376358, 0.04056627303361893, 0.07904838770627975, -0.06350173056125641, 0.036567240953445435, -0.022027628496289253, 0.07078643143177032, -0.04911943897604942, 0.07224114239215851, 0.09873174875974655, 0.09077904373407364, 0.01918254978954792, 0.06441287696361542, -0.055515583604574203, 0.09757357835769653, -0.08858566731214523, 0.018643725663423538, 0.053014885634183884, 0.00664991419762373, -0.10532554984092712, 0.04616064578294754, -0.1043853908777237, 0.04755004867911339, -0.03359074518084526, 0.11164864897727966, 0.06907203793525696, -0.08971088379621506, -0.06069185957312584, -0.025967387482523918, 0.09520060569047928, -0.08418023586273193, 0.08419424295425415, 0.0929989293217659, -0.11806531995534897, -0.09352298080921173, -0.06758473813533783, 0.03563099354505539, -0.11646278947591782, -0.03517023101449013, 0.005398379638791084, 0.052059028297662735, 0.07966319471597672, -0.08287404477596283, 0.030537661164999008, -0.07386395335197449, -0.09409405291080475, 0.029793504625558853, 0.11417566239833832, -0.06310770660638809, -0.14537425339221954, 0.059021685272455215, 0.04456289857625961, -0.03843654319643974, -0.1003769263625145, 0.06965622305870056, 0.049899742007255554, 0.09289412945508957, -0.003301457269117236, 0.039032164961099625, -0.03210588917136192, 0.017019081860780716, -0.03291096165776253, -0.013673129491508007, -0.11702100187540054, 0.013654332607984543, 0.06603086739778519, 0.06094575300812721, 0.05714448168873787, -0.03582089766860008, 0.08297594636678696, -0.08059731125831604, 0.08113465458154678, -0.011951692402362823, -0.06118687987327576, -0.06755289435386658, -0.11773059517145157, -0.10251124948263168, -0.09322988241910934, 0.11079277098178864, 0.033149149268865585, -0.12021225690841675, -0.09524311870336533, 0.0860968604683876, 0.06798405945301056, -0.05083918944001198, -0.032184407114982605, -0.05106436088681221, 0.08553510904312134, 0.08136951178312302, -0.042232755571603775, 0.07083960622549057, -0.07633090019226074, -0.016634304076433182, -0.02941759116947651, -0.1496891975402832, 0.010895764455199242, 0.11965595930814743, -7.519207429140806e-06, 0.043780092149972916, -0.15889446437358856, 0.09314874559640884, -0.04718410223722458, -0.020254386588931084, -0.018896957859396935, -0.07790439575910568, 0.043191708624362946, -0.09868716448545456, 0.011488190852105618, -0.10126740485429764, -0.012818525545299053, 0.04294411092996597, 0.09139275550842285, -0.037881653755903244, -0.0853530541062355, 0.09429370611906052, -0.15445542335510254, -0.08500050008296967, 0.02216409705579281, 0.036543212831020355], "bias": [-0.007104420103132725, -0.06966141611337662, 0.11174521595239639, 0.08230949193239212, -0.04930930212140083, 0.027230313047766685, -0.08406311273574829, 0.08484739810228348, -0.04336250573396683, -0.1233149990439415, -0.03216472268104553, 0.0691135823726654, 0.015423653647303581, -0.05616501346230507, -0.045072298496961594, -0.0003470725496299565, 0.12280751019716263, -0.027355056256055832, 0.032202593982219696, 0.10721176117658615, 0.0825117751955986, 0.05665665864944458, -0.08654627203941345, -0.1023801863193512, -0.11288446933031082, 0.06776987016201019, -0.0909607782959938, 0.08186119794845581, -0.052685968577861786, -0.11394327878952026, 0.10325280576944351, -0.00045478809624910355, -0.07184137403964996, -0.009836073964834213, -0.09093525260686874, -0.15164540708065033, 0.04680917412042618, 0.002291544107720256, -0.09185760468244553, 0.11301559209823608, -0.06933195143938065, -0.10059604048728943, 0.13703376054763794, 0.04913315549492836, -0.07477658241987228, 0.012187724933028221, -0.10570038855075836, 0.13199838995933533, -0.06099563464522362, 0.01585361175239086, -0.11002259701490402, 0.07087792456150055, -0.1087222471833229, -0.061979614198207855, -0.14187565445899963, -0.07350045442581177, 0.041954945772886276, -0.14090164005756378, -0.14698246121406555, -0.02343018911778927, 0.03497966751456261, -0.002229714300483465, -0.05872669070959091, -0.03715738281607628, -0.012158479541540146, -0.05733884125947952, -0.09205891191959381, 0.08753922581672668, 0.01889662630856037, -0.14013683795928955, -0.0510665699839592, -0.09242633730173111, -0.066709503531456, 0.07564941048622131, 0.02475886605679989, 0.09867004305124283, -0.0004474290763027966, -0.0815015509724617, 0.042432256042957306, -0.1333089917898178, -0.17089171707630157, 0.15545298159122467, 0.039763130247592926, 0.0950956642627716, 0.14469990134239197, 0.06877683848142624, -0.0430554635822773, -0.005848235916346312, 0.027833931148052216, -0.05419561266899109, -0.0060478984378278255, -0.10921164602041245, -0.06169182434678078, 0.122007817029953, 0.015232660807669163, -0.10469955205917358, 0.06354615092277527, -0.03812512010335922, -0.053964562714099884, 0.09092764556407928, -0.1421266496181488, -0.09481444209814072, 0.10835563391447067, -0.0768580287694931, -0.03226304054260254, 0.13640733063220978, 0.011583004146814346, -0.1644362062215805, -0.16306966543197632, -0.07842764258384705, -0.0033455113880336285, -0.09389779716730118, -0.02921403758227825, 0.003674235427752137, -0.15142183005809784, -0.12334044277667999, 0.03996090218424797, 0.01314719207584858, -0.08356527239084244, -0.13330911099910736, 0.11685249209403992, 0.08906982094049454, -0.04019752889871597, 0.11417526006698608, 0.1057049036026001, 0.13700710237026215, -0.05208997800946236, -0.04142308235168457]}, {"kind": "relu"}, {"kind": "affine", "scale": [3.2363524436950684, 2.6663782596588135, 2.6750950813293457, 2.2425243854522705, 2.592353582382202, 2.631514310836792, 3.2439396381378174, 2.3095827102661133, 3.052953004837036, 2.1648850440979004, 3.3375015258789062, 2.158782958984375, 3.184490442276001, 2.877568006515503, 3.260652780532837, 2.605375051498413, 2.665189504623413, 2.5300686359405518, 2.4400475025177, 2.6921143531799316, 2.4584507942199707, 3.011611223220825, 3.218940019607544, 3.249680519104004, 3.1014177799224854, 3.7223613262176514, 3.083142042160034, 2.4726943969726562, 2.4518380165100098, 2.504523754119873, 2.566879987716675, 2.9522652626037598, 3.169621706008911, 3.4639463424682617, 2.9126996994018555, 3.9264533519744873, 2.5517845153808594, 2.6425907611846924, 2.627274990081787, 2.782290458679199, 3.0998029708862305, 3.499835252761841, 2.1227807998657227, 2.4238932132720947, 2.883061408996582, 2.754387140274048, 3.0339574813842773, 2.3744056224823, 3.1281628608703613, 3.0973398685455322, 3.7152087688446045, 2.7090892791748047, 2.9126851558685303, 3.2367746829986572, 4.229969024658203, 3.050363063812256, 2.9768168926239014, 2.6429970264434814, 3.532144069671631, 2.8560423851013184, 2.648979425430298, 2.642061471939087, 2.4900479316711426, 3.0618324279785156, 2.839120864868164, 3.5958526134490967, 2.964454412460327, 2.3312885761260986, 3.2127771377563477, 2.80761456489563, 2.658622980117798, 2.661097526550293, 2.4118878841400146, 2.659968137741089, 2.562321186065674, 3.1069934368133545, 2.5915067195892334, 3.2289960384368896, 2.94315505027771, 3.146461248397827, 2.7885520458221436, 2.511016845703125, 2.8912460803985596, 2.645728349685669, 2.343740224838257, 2.778918743133545, 2.7069954872131348, 2.5556576251983643, 2.74849796295166, 2.7066245079040527, 2.589895725250244, 2.9126460552215576, 3.480708599090576, 2.500976324081421, 2.4057559967041016, 3.693669557571411, 2.3836281299591064, 2.569688081741333, 3.2293009757995605, 2.2684226036071777, 2.8507912158966064, 3.2541637420654297, 2.346707820892334, 3.224384307861328, 2.867396354675293, 2.824174165725708, 2.5756168365478516, 2.9294753074645996, 3.2975611686706543, 3.3395025730133057, 3.036066770553589, 3.1321983337402344, 2.231060266494751, 3.045844793319702, 3.0870862007141113, 2.7141597270965576, 2.804422378540039, 2.8191020488739014, 2.8475120067596436, 2.8976566791534424, 2.71646785736084, 2.665703058242798, 2.8853321075439453, 2.3674726486206055, 2.7567079067230225, 2.3936243057250977, 2.794254779815674, 3.1206037998199463], "bias": [-0.6541354060173035, -0.5913528800010681, -0.7503173351287842, -0.7141716480255127, -0.5871474146842957, -0.7002246975898743, -0.620952844619751, -0.6803003549575806, -0.6599624156951904, -0.4974461495876312, -0.673351526260376, -0.6877874732017517, -0.708714485168457, -0.5893958806991577, -0.5839976072311401, -0.6164035201072693, -0.7517240643501282, -0.6101162433624268, -0.6522558331489563, -0.7422475814819336, -0.7227622270584106, -0.716817319393158, -0.6298306584358215, -0.5589852929115295, -0.5754742622375488, -0.877377986907959, -0.47477248311042786, -0.6947540640830994, -0.5628945231437683, -0.5264551043510437, -0.6858208775520325, -0.622121274471283, -0.5944350361824036, -0.5836083292961121, -0.5306457281112671, -0.5196057558059692, -0.7219349145889282, -0.6627911329269409, -0.5354093909263611, -0.7401092052459717, -0.6284471154212952, -0.47853681445121765, -0.7996645569801331, -0.7071781158447266, -0.5539669394493103, -0.6426207423210144, -0.5730867981910706, -0.7822734713554382, -0.6422898173332214, -0.668665885925293, -0.583400309085846, -0.7331510782241821, -0.4979332685470581, -0.5621029138565063, -0.5116616487503052, -0.5562254786491394, -0.7325959205627441, -0.5317301750183105, -0.5207415223121643, -0.6234442591667175, -0.6704205274581909, -0.6409307718276978, -0.6481310725212097, -0.6669080853462219, -0.6138155460357666, -0.5912464261054993, -0.5751203298568726, -0.7739378809928894, -0.6849808692932129, -0.5550851225852966, -0.5737801790237427, -0.6352256536483765, -0.6442441940307617, -0.689278244972229, -0.620728075504303, -0.8242356181144714, -0.6592860221862793, -0.5831243991851807, -0.7025780081748962, -0.5942872166633606, -0.5212631225585938, -0.8053516149520874, -0.6791433095932007, -0.7548424005508423, -0.8597499132156372, -0.667700469493866, -0.5954244136810303, -0.6437042355537415, -0.691381573677063, -0.6067332029342651, -0.6978515386581421, -0.5442137122154236, -0.633054256439209, -0.8150293827056885, -0.6606056690216064, -0.5483357310295105, -0.6148407459259033, -0.6959432363510132, -0.639670729637146, -0.6660363078117371, -0.5489847660064697, -0.5336363911628723, -0.8056857585906982, -0.5805642604827881, -0.6625353097915649, -0.8061856627464294, -0.6948776245117188, -0.5178519487380981, -0.5105556845664978, -0.5587823390960693, -0.6913115382194519, -0.5841549038887024, -0.6334996819496155, -0.6828315854072571, -0.5243370532989502, -0.5630543231964111, -0.680335283279419, -0.6546170115470886, -0.5449312925338745, -0.5563636422157288, -0.7187579870223999, -0.7437360286712646, -0.6503018736839294, -0.7472223043441772, -0.807701826095581, -0.8012902140617371, -0.6423361897468567, -0.625968337059021]}, {"kind": "linear", "input": 128, "output": 128, "weight": [0.08012440800666809, -0.006574355065822601, 0.0324893444776535, -0.041553616523742676, -0.028048640117049217, -0.09900863468647003, -0.03205367550253868, -0.005016002804040909, -0.0049643260426819324, 0.07260093837976456, 0.00819160882383585, -0.022159572690725327, 0.04591332748532295, -0.05845395848155022, -0.00624041398987174, 0.03335971012711525, -0.019915470853447914, -0.04403161630034447, -0.04378121718764305, 0.05150139704346657, -0.07688981294631958, -0.00454730074852705, 0.03371143713593483, -0.03004598245024681, -0.017714623361825943, 0.05485604330897331, 0.06286783516407013, -0.049882568418979645, 0.009952574968338013, -0.0537693053483963, -0.05404195189476013, 0.09447403252124786, -0.0038145179860293865, -0.020217787474393845, 0.06165071576833725, 0.03843183442950249, -0.08763106912374496, -0.05126870423555374, 0.025673290714621544, -0.02239636331796646, -0.0609632208943367, -0.03719554468989372, -0.011956467293202877, 0.017535744234919548, -0.01454346813261509, 0.015097338706254959, 0.05972737818956375, 0.01220750156790018, 0.1151302307844162, -0.03645291179418564, 0.019749922677874565, 0.008680545724928379, -0.05180719122290611, -0.06469639390707016, -0.020601678639650345, -0.021962005645036697, -0.07734405249357224, -0.10313493013381958, -0.003973086830228567, 0.08082491159439087, 0.02255709283053875, -0.011326301842927933, -0.01148952730000019, -0.043149590492248535, -0.04053273797035217, 0.012203843332827091, -0.023946307599544525, 0.05978759378194809, -0.04928804561495781, -0.06273060292005539, -0.03423306718468666, 0.005484364926815033, 0.04817833751440048, -0.05196006968617439, -0.08675768226385117, 0.02565006911754608, 0.028312671929597855, 0.0120353102684021, 0.007882286794483662, 0.055173274129629135, -0.030226122587919235, 0.04967718943953514, -0.04754327982664108, 0.11332312226295471, 0.09038126468658447, -0.004952593706548214, -0.04880950599908829, 0.08068933337926865, -0.022533588111400604, 0.003402398433536291, 0.09975779056549072, -0.0562698096036911, -0.0330447256565094, 0.03835291787981987, 0.027949871495366096, -0.009749971330165863, -0.05926148593425751, 0.08328887075185776, -0.04710753634572029, 0.07192442566156387, -0.04988444596529007, -0.03515343368053436, -0.06124940887093544, -0.0452357716858387, 0.011382597498595715, 0.0661470964550972, 0.0732940211892128, 0.09496919065713882, 0.04438679665327072, -0.11347775161266327, -0.0342000387609005, -0.062410660088062286, -0.04248403385281563, -0.0997939333319664, -0.007515203673392534, -0.08346109092235565, 0.01618475466966629, 0.03651318699121475, 0.009588469751179218, -0.07289442420005798, -0.0019265442388132215, -0.000290687195956707, -0.057615410536527634, 0.005230930168181658, 0.1126648411154747, 0.08392339944839478, -0.006609026808291674, 0.008056042715907097, 0.03417661041021347, -0.027625877410173416, 0.042536161839962006, 0.004555906169116497, -0.09267499297857285, -0.036626581102609634, -0.08347505331039429, 0.08433623611927032, -0.06497568637132645, -0.006459705997258425, -0.0036805449053645134, 0.07918767631053925, 0.01738397404551506, 0.049810491502285004, -0.06691616773605347, 0.07599784433841705, 0.08373034000396729, -0.014250727370381355, 0.053945351392030716, -0.03759178891777992, 0.00021831376943737268, 0.05363462492823601, -0.060390833765268326, 0.110818050801754, -0.0019876270089298487, 0.036992426961660385, 0.0253767017275095, -0.028195075690746307, -0.061951350420713425, 0.024718742817640305, 0.051804158836603165, -0.019364917650818825, -0.08079291880130768, 0.01750301942229271, 0.05428970232605934, 0.058265652507543564, -0.08204464614391327, 0.026776732876896858, 0.03531217575073242, -0.09783928841352463, 0.028998855501413345, -0.049674734473228455, -0.12120974808931351, -0.032991308718919754, 0.022492581978440285, -0.0025048493407666683, -0.0670037493109703, -0.07283497601747513, -0.05097595229744911, 0.02002931386232376, 0.05860931798815727, -0.07785755395889282, -0.09785397350788116, 0.061438847333192825, -0.06734747439622879, -0.09894730895757675, 0.05354743078351021, -0.10344620794057846, -0.007065360434353352, -0.030354836955666542, -0.07710817456245422, 0.05165243521332741, 0.02506260573863983, -0.02719889022409916, -0.019559677690267563, 0.015708979219198227, -0.07696676254272461, -0.0230732262134552, 0.09304089099168777, 0.06878668814897537, -0.02926899492740631, -0.06509822607040405, 0.07141686975955963, -0.03265168517827988, 0.007114902138710022, -0.04986535757780075, -0.029410986229777336, -0.07712947577238083, -0.00034209428122267127, -0.00970755610615015, -0.025611644610762596, 0.016790777444839478, 0.005135667975991964, -0.03188437595963478, 0.0635470449924469, -0.021750953048467636, 0.0022189710289239883, -0.05218782275915146, -0.039635736495256424, -0.028018280863761902, 0.04806074500083923, 0.0600467287003994, -0.06401033699512482, 0.008729293011128902, -0.01118467003107071, -0.017521122470498085, -0.014939770102500916, -0.0651572048664093, 0.018918756395578384, 0.029141312465071678, -0.006883617490530014, -0.05180799588561058, -0.06367990374565125, 0.0026728990487754345, 0.09904446452856064, 0.016035307198762894, 0.08675048500299454, 0.05390128493309021, 0.06695093214511871, -0.04001891613006592, 0.02922242507338524, -0.025257552042603493, 0.07856804132461548, -0.10229950398206711, 0.025498349219560623, -0.00552820460870862, -0.04896653816103935, 0.0037914810236543417, -0.005314864218235016, -0.06668667495250702, 0.006765233818441629, 0.0428127720952034, 0.045680802315473557, -0.05808211490511894, 0.03824269771575928, -0.019575556740164757, -0.07260988652706146, -0.031967177987098694, -0.09038151055574417, 0.07521941512823105, 0.02449699304997921, 0.08908049017190933, -0.039551444351673126, -0.07762282341718674, -0.08299923688173294, 0.039908841252326965, 0.03659068048000336, -0.040632110089063644, 0.02219129353761673, -0.061670731753110886, -0.1093798503279686, 0.017854757606983185, -0.043887749314308167, -0.11461350321769714, 0.042212363332509995, 0.023400604724884033, 0.05634651705622673, 0.018266627565026283, -0.06239287182688713, 0.005049318540841341, 0.03415524214506149, -0.008813108317553997, 0.010226551443338394, -0.07973417639732361, 0.031884822994470596, 0.008909297175705433, -0.013542186468839645, -0.020136255770921707, 0.038702234625816345, -0.1035386249423027, -0.08677206188440323, -0.09672554582357407, -0.056117452681064606, 0.013446370139718056, 0.013288315385580063, 0.06735176593065262, -0.04546833783388138, -0.014117252081632614, 0.00409309659153223, -0.08438525348901749, -0.034783340990543365, 0.09195555746555328, -0.028721394017338753, -0.09020737558603287, 0.07353675365447998, -0.06389541923999786, -0.06058449298143387, 0.0430336557328701, -0.07975072413682938, 0.04283210262656212, -0.07333924621343613, -0.07098721712827682, 0.03411806374788284, -0.09384693205356598, 0.0011156720574945211, -0.0051873549818992615, -0.0025733502116054296, -0.012200837023556232, -0.02497490681707859, 0.06078464165329933, 0.08497171849012375, 0.01903279684484005, 0.08436011523008347, -0.06407757103443146, -0.05725069344043732, -0.04995162412524223, 0.02423178218305111, -0.06943746656179428, -0.03434186428785324, -0.033973246812820435, 0.06505949050188065, -0.10756190866231918, -0.006987215485423803, 0.08904514461755753, -0.08667055517435074, 0.08152949064970016, 0.020537633448839188, 0.047216713428497314, 0.027686769142746925, -0.08292969316244125, -0.058087076991796494, 0.02696060948073864, -0.0634327158331871, 0.0643983855843544, -0.07946240901947021, 0.034991100430488586, 0.011126228608191013, -0.0445464625954628, 0.024736491963267326, 0.009913734160363674, 0.014840170741081238, 0.07069043070077896, -0.0621955581009388, 0.07744046300649643, -0.02543575130403042, 0.027662402018904686, -0.0385114811360836, -0.038802094757556915, 0.03506898134946823, -0.06213022395968437, -0.010512000881135464, -0.024892808869481087, -0.05884823203086853, 0.0055052749812603, -0.06737672537565231, -0.010116822086274624, -0.049190275371074677, -0.01597803458571434, 0.0711996927857399, -0.045038994401693344, 0.02812035009264946, -0.09188052266836166, -0.055940426886081696, 0.04937664791941643, -3.7239049561321735e-05, 0.032891057431697845, 0.053559496998786926, -0.025618571788072586, 0.040429919958114624, 0.007259775418788195, 0.10556425154209137, -0.0248152706772089, -0.029605410993099213, -0.07532193511724472, 0.0010484314989298582, 0.05268637835979462, 0.051914557814598083, 0.030085377395153046, 0.08885529637336731, 0.03801719471812248, -0.011779546737670898, 0.019507981836795807, 0.0009747949661687016, 0.07026292383670807, 0.07335567474365234, 0.0447450652718544, 0.028776252642273903, 0.03332211449742317, -0.005836385302245617, -0.046377070248126984, 0.046385738998651505, -0.10649830102920532, -0.04280023276805878, -0.056640394032001495, 0.036106325685977936, 0.034739937633275986, -0.05451524257659912, -0.020215654745697975, -0.0729636400938034, 0.09037360548973083, -0.10638191550970078, -0.015697810798883438, -0.010544919408857822, -0.06068076938390732, 0.03618443012237549, -0.03746030107140541, -0.06438051909208298, -0.03474387153983116, -0.047000378370285034, -0.009487945586442947, 0.043264057487249374, 0.08995130658149719, -0.05333539843559265, 0.056696757674217224, -0.0749875158071518, 0.03423534333705902, -0.005665004253387451, 0.05061384662985802, -0.015059703961014748, -0.058942295610904694, 0.05564777925610542, -0.10334904491901398, -0.02461601234972477, 0.007254011929035187, -0.03591812029480934, 0.01281549222767353, -0.046667296439409256, -0.040147051215171814, -0.004304212052375078, 0.07194223254919052, 0.07017764449119568, 0.06468290835618973, -0.08371391892433167, -0.026651356369256973, 0.04443502053618431, -0.04057871922850609, -0.057896699756383896, 0.019613096490502357, 0.02610151655972004, 0.04416847229003906, -0.0417163223028183, -0.10279063135385513, -0.06706590205430984, -0.0609816238284111, 0.05720899999141693, -0.06563157588243484, -0.012298187240958214, 0.060608867555856705, -0.09124340862035751, 0.015354827046394348, -0.04103793948888779, 0.04469989985227585, -0.07521284371614456, -0.02234780415892601, 0.08719811588525772, 0.09368914365768433, -0.0672823116183281, -0.06093396618962288, -0.046697527170181274, -0.013456449843943119, -0.052652787417173386, 0.028072044253349304, -0.04087395593523979, 0.04823177680373192, -0.013344289734959602, -0.06993570178747177, 0.07258807122707367, -0.07436554878950119, -0.05092700570821762, -0.06514263898134232, -0.008144531399011612, -0.07843223959207535, -0.06329872459173203, -0.00563622685149312, 0.06373517215251923, 0.08588680624961853, -0.015079852193593979, 0.002174904104322195, 0.028533603996038437, 0.008689540438354015, 0.09560080617666245, 0.08592945337295532, 0.09699653089046478, -0.01995287463068962, -0.031671080738306046, -0.007298717740923166, 0.01944148540496826, -0.0811816155910492, 0.03636566549539566, -0.06493056565523148, 0.030394719913601875, 0.06520108133554459, -0.02367902733385563, 0.0398104153573513, -0.057879090309143066, -0.043108128011226654, -0.0431661531329155, 0.06371526420116425, -0.039913784712553024, 0.05950319394469261, -0.0775114893913269, 0.01680549792945385, -0.008030175231397152, -0.022082308307290077, 0.06872989237308502, -0.0672101229429245, 0.0075806560926139355, -0.004807607736438513, -0.05695989355444908, -0.00977887213230133, 0.069565549492836, -0.0011221995810046792, 0.014167938381433487, 0.04920940101146698, -0.007441876921802759, 0.08714812248945236, 0.059436872601509094, -0.07906925678253174, -0.008106090128421783, -0.017038676887750626, 0.02361873909831047, -0.0629430040717125, -0.03327956423163414, 0.08160841464996338, -0.024228718131780624, -0.02387811802327633, -0.05578674003481865, -0.09031277149915695, 0.04017604887485504, -0.061443034559488297, 0.09015247970819473, 0.06912129372358322, -0.05861501768231392, 0.0788515955209732, -0.007210780866444111, 0.07926439493894577, 0.0205670278519392, -0.08157055079936981, -0.05180542543530464, -0.02900123968720436, 0.05308317020535469, -0.07604847848415375, 0.0224989615380764, 0.07505398988723755, 0.021107496693730354, 0.014417293481528759, 0.04282187670469284, -0.046771783381700516, -0.0627458393573761, 0.0226706862449646, -0.03148256987333298, -0.03734589368104935, -0.04708139970898628, 0.03610550984740257, 0.033168379217386246, -0.05571576952934265, -0.04519486799836159, 0.054071635007858276, 0.08526551723480225, -0.015598984435200691, -0.014053238555788994, -0.04743548110127449, 0.05341499298810959, -0.04988465458154678, -0.08600828796625137, 0.07705681025981903, 0.03450499847531319, -0.026320863515138626, -0.06282919645309448, 0.008967938832938671, -0.08979272097349167, 0.03350057452917099, 0.023799719288945198, -0.010211261920630932, 0.03675730153918266, -0.023420773446559906, 0.009753305464982986, -0.06680110841989517, -0.07331930845975876, 0.015453072264790535, -0.06026604771614075, -0.052410025149583817, -0.015435362234711647, 0.08188758790493011, -0.03747342899441719, 0.018982017412781715, 0.03390342742204666, 0.012091200798749924, -0.0435674712061882, -0.10890386998653412, 0.013253909535706043, 0.04019995033740997, -0.09862559288740158, -0.040639813989400864, 0.030075425282120705, -0.08889767527580261, 0.04493144527077675, -0.09228415042161942, -0.03846899792551994, -0.05262603238224983, 0.015174689702689648, 0.0206929761916399, -0.02823665551841259, 0.006039427127689123, -0.023451222106814384, -0.008587430231273174, -0.013023504987359047, -0.014235190115869045, 0.03642060607671738, 0.003973756916821003, 0.025007598102092743, -0.049021873623132706, 0.06811206787824631, -0.022550422698259354, 0.09941980242729187, -0.06458180397748947, -0.05863296613097191, -0.01483902707695961, 0.013927138410508633, 0.028805889189243317, 0.06796933710575104, -0.03145161271095276, -0.12563352286815643, 0.05504761263728142, 0.041996393352746964, 0.023807357996702194, -0.07713791728019714, 0.021133968606591225, -0.020285941660404205, -0.00631018215790391, -0.07109759747982025, -0.06349378824234009, -0.06720717996358871, 0.02188650704920292, -0.04047264903783798, -0.059382177889347076, -0.002211394952610135, -0.03961814567446709, 0.038293901830911636, 0.021076807752251625, -0.04624275863170624, 0.06929868459701538, -0.007223804946988821, 0.0062265037558972836, 0.05455688014626503, 0.012872780673205853, -0.08959245681762695, -0.04053862392902374, -0.06957228481769562, -0.07145947217941284, -0.06325990706682205, 0.04602640122175217, 0.039863452315330505, 0.027896754443645477, -0.02805718220770359, -0.051287081092596054, -0.0017255691345781088, 0.04123399779200554, 0.029040634632110596, 0.08636051416397095, -0.0975361242890358, -0.07571344822645187, -0.03803567215800285, 0.05323045328259468, -0.06714984774589539, -0.06931551545858383, -0.0006674144533462822, -0.08223021775484085, -0.025780556723475456, -0.0750974491238594, -0.005994671955704689, -0.00877333153039217, 0.06421621143817902, 0.05065377056598663, 0.06791847944259644, 0.08074097335338593, 0.035863298922777176, -0.06014786660671234, 0.06455179303884506, 0.016802797093987465, -0.08115547150373459, -0.07454994320869446, -0.08160827308893204, -0.02677335776388645, 0.0439629927277565, 0.048174839466810226, 0.0574500635266304, -0.005847270134836435, -0.022983849048614502, -0.06918312609195709, 0.06267210096120834, -0.03312603011727333, -0.06797963380813599, 0.0485135018825531, 0.04994333162903786, 0.10097914934158325, -0.04940483346581459, 0.04510054364800453, 0.055125195533037186, -0.049787286669015884, 0.07635509967803955, -0.0419842004776001, -0.08194544911384583, -0.05386357009410858, 0.07614394277334213, -0.10906128585338593, 0.044771458953619, 0.0517737977206707, 0.08037005364894867, 0.012484521605074406, 0.07998184114694595, 0.06031278148293495, -0.027728578075766563, -0.01683119870722294, 0.0809713676571846, -0.04283768683671951, 0.004250153433531523, -0.12873908877372742, -0.04393627867102623, -0.0248667411506176, -0.0064030797220766544, 0.028286172077059746, 0.0037083004135638475, 0.008663427084684372, 0.02621445059776306, 0.019007330760359764, -0.006484161596745253, 0.024064786732196808, -0.01613309420645237, -0.052897922694683075, 0.04412668198347092, -0.01694386824965477, 0.07917850464582443, 0.07369577884674072, 0.03248879685997963, 0.037138957530260086, 0.07869485020637512, 0.025712108239531517, 0.07372158765792847, 0.05222316086292267, -0.12344349175691605, 0.03861549124121666, -0.004575320053845644, -0.013498485088348389, 0.07945229858160019, 0.06406773626804352, -0.05031860992312431, -0.022733217105269432, 0.03675045445561409, 0.06776224821805954, 0.02868761122226715, -1.962226815521717e-05, 0.0517570897936821, -0.04041676968336105, -0.04092220962047577, -0.11610342562198639, 0.05464072525501251, -0.0009471382945775986, 0.038154762238264084, -0.0912916362285614, -0.11724581569433212, -0.05477257817983627, -0.06093427166342735, -0.06552896648645401, -0.04938990995287895, 0.09206913411617279, -0.02437887154519558, 0.06980656832456589, -0.025660771876573563, 0.050368744879961014, 0.08608852326869965, -0.10078215599060059, 0.009208683855831623, -0.08115457743406296, 0.016316743567585945, -0.0029262739699333906, 0.02083149552345276, -0.02173578552901745, -0.056021347641944885, 0.020821522921323776, 0.06470305472612381, -0.07846368849277496, -0.04862331226468086, 0.01215960830450058, 0.008855351246893406, -0.00574845215305686, 0.03363754227757454, 0.021685482934117317, -0.05554565414786339, -0.09795250743627548, 0.0015128833474591374, 0.06140865013003349, -0.05633487552404404, 0.05502834916114807, 0.04374400153756142, -0.055286601185798645, -0.03602290153503418, -0.0003019275900442153, -0.058038923889398575, 0.06620416790246964, 0.04001989588141441, 0.07362646609544754, 0.00782909244298935, -0.06586931645870209, 0.010678705759346485, -0.06425148248672485, 0.04494573548436165, 0.025903815403580666, -0.0012082925532013178, -0.06653536856174469, -0.03651147335767746, 0.026074891909956932, 0.01577259972691536, -0.013800901360809803, 0.055812008678913116, -0.04750233143568039, -0.049615420401096344, 0.006476273760199547, 0.07349283993244171, -0.006846214644610882, -0.059040751308202744, 0.05297011509537697, -0.0280088409781456, -0.018199849873781204, 0.045061178505420685, -0.07399100810289383, -0.05045764148235321, 0.056158699095249176, -0.06468534469604492, -0.0331951305270195, 0.031555481255054474, 0.06240333616733551, 0.07336684316396713, 0.032342132180929184, -0.013300842605531216, 0.024949699640274048, -0.019625380635261536, 0.09587792307138443, -0.05647512897849083, -0.009661944583058357, -0.03243042156100273, -0.004826173186302185, -0.0249212384223938, -0.08951918035745621, 0.020404024049639702, -0.0004010643169749528, 0.07016360014677048, -0.073200523853302, -0.09665761888027191, 0.009492313489317894, 0.037176478654146194, -0.003215920412912965, -0.08214141428470612, -0.07547120749950409, 0.05894992873072624, 0.05740305408835411, -0.02930191159248352, -0.027433179318904877, -0.025446487590670586, 0.03819466754794121, 0.09505974501371384, 0.10512582212686539, -0.04119883105158806, -0.005536934360861778, -0.011794086545705795, -0.011052408255636692, -0.09869135171175003, 0.010961968451738358, 0.074985571205616, 0.10674795508384705, 0.10631384700536728, -0.0041999719105660915, -0.05791035667061806, -0.010991699993610382, 0.042669281363487244, 0.0970047265291214, -0.002162517746910453, 0.03266448900103569, -0.022672763094305992, 0.0343332439661026, -0.05841106176376343, 0.04261646419763565, 0.04360390082001686, -0.05214650183916092, -0.09533029794692993, -0.084995798766613, 0.03455144539475441, -0.061449192464351654, -0.08907919377088547, 0.01802556775510311, -0.009768350049853325, -0.00031230185413733125, 0.02353951707482338, -0.0486256405711174, -0.07749076932668686, 0.04461623728275299, -0.050210896879434586, -0.052952881902456284, -0.036150068044662476, -0.05162776634097099, 0.08181961625814438, -0.08385934680700302, -0.057971637696027756, 0.010357100516557693, 0.030736949294805527, -0.10988537967205048, -0.04176926985383034, 0.011502528563141823, -0.03129957243800163, -0.09384371340274811, -0.003328781109303236, -0.003175254911184311, 0.02292199805378914, 0.05160418897867203, 0.056590061634778976, 0.0655825212597847, 0.05079498514533043, -0.04557746276259422, -0.04864828288555145, 0.055040813982486725, -0.07388697564601898, -0.045418158173561096, 0.0045155733823776245, -0.08493071794509888, -0.032994166016578674, 0.06480835378170013, 0.06855779141187668, -0.025598151609301567, -0.0002238539746031165, -0.05917128175497055, -0.08983123302459717, 0.04039990156888962, 0.07772719115018845, -0.06716042757034302, -0.0536058135330677, 0.06072595342993736, -0.07689864188432693, 0.04878482222557068, 0.0017692521214485168, -0.023062124848365784, 0.01005508378148079, -0.11333895474672318, -0.0620613731443882, 0.0861135795712471, -0.03849339485168457, 0.001956618158146739, 0.02560892514884472, -0.018052587285637856, 0.09260707348585129, -0.03599698096513748, -0.048815641552209854, 0.07742088288068771, -0.006178470328450203, 0.09445740282535553, 0.07103417813777924, -0.0789431780576706, 0.01729140616953373, 0.055577848106622696, 0.09706460684537888, -0.03664637356996536, 0.04823046922683716, 0.008422846905887127, 0.0178826954215765, -0.06247652694582939, -0.06228657066822052, -0.03960151970386505, -0.04739975929260254, -0.08860461413860321, 0.07251454889774323, -0.01687612757086754, -0.037013426423072815, -0.0787862241268158, -0.024867255240678787, -0.048921745270490646, 0.050095848739147186, 0.017208125442266464, -0.042603347450494766, -0.0086020827293396, 0.000724534154869616, 0.04878073185682297, -0.08288723975419998, 0.08737558871507645, -0.09579025208950043, -0.040391705930233, -0.052157673984766006, 0.07365535199642181, -0.04679229110479355, -0.042456790804862976, 0.07149425894021988, -0.058047544211149216, -0.019563887268304825, -0.04698406532406807, 0.060966454446315765, 0.047010570764541626, 0.007432722020894289, 0.05062125250697136, 0.07638247311115265, -0.01884176768362522, -0.008939557708799839, -0.03657784312963486, 0.0524616464972496, 0.02581503987312317, -0.03385505452752113, -0.032405924052000046, -0.015527469106018543, 0.005891896318644285, 0.06857997924089432, -0.09111681580543518, 0.08562760055065155, -0.12669505178928375, 0.04826423153281212, -0.038085900247097015, -0.005184529349207878, 0.048339031636714935, 0.02508465200662613, -0.005738997366279364, -0.0458880290389061, -0.004118096549063921, 5.23347407579422e-05, -0.03638950735330582, 0.06407125294208527, -0.005813525523990393, -0.0695851519703865, 0.06970233470201492, -0.057536765933036804, 0.016676194965839386, -0.0860385075211525, 0.08894694596529007, -0.06733030825853348, 0.030139930546283722, -0.06753048300743103, 0.06065702810883522, 0.08212428539991379, 0.019308477640151978, -0.0795944407582283, -0.030684666708111763, -0.044071014970541, -0.033430829644203186, -0.0029481302481144667, -0.08148068934679031, 0.010357022285461426, -0.02182181552052498, 0.02268141694366932, 0.08601298928260803, 0.0015085979830473661, -0.08163236826658249, -0.07735031843185425, -0.0664013996720314, -0.039637140929698944, -0.07288161665201187, 0.0130237378180027, -0.022934213280677795, -0.05838308855891228, -0.04175051674246788, -0.07308141142129898, -0.07245151698589325, -0.09886036813259125, -0.02900441735982895, -0.020593035966157913, 0.036338455975055695, 0.018929168581962585, -0.047502774745225906, 0.04917261004447937, 0.028862128034234047, 0.05069602653384209, 0.03749214857816696, 0.06373485922813416, -0.09275788813829422, 0.04849490895867348, 0.0591067411005497, -0.07761996239423752, -0.03640051558613777, 0.005148407071828842, 0.0772666186094284, -0.015204457566142082, 0.011671929620206356, -0.025876369327306747, -0.0708463191986084, -0.0007439838373102248, 0.0883515402674675, 0.023932693526148796, -0.051975760608911514, 0.05254267528653145, 0.04966054484248161, -0.017337867990136147, 0.05957546457648277, 0.03764606639742851, -0.06396647542715073, -0.03811872750520706, -0.0301925390958786, -0.026666419580578804, 0.06271558254957199, 0.018390128389000893, -0.00868241861462593, -0.03565393388271332, 0.04790235310792923, 0.008566014468669891, 0.08333143591880798, 0.051642876118421555, 0.05776028335094452, -0.059316929429769516, 0.036279547959566116, 0.03657345846295357, -0.01725287176668644, 0.04858345538377762, -0.07802622765302658, 0.10133689641952515, -0.018147168681025505, -0.048482269048690796, 0.0031875544227659702, -0.03169400990009308, -0.03327031806111336, 0.02716255746781826, -0.011029009707272053, -0.013698309659957886, 0.04542553797364235, -0.02153085730969906, 0.012081797234714031, 0.030207611620426178, -0.045009713619947433, 0.018688837066292763, 0.0762270987033844, 0.02083529159426689, 0.031061168760061264, 0.0607341043651104, 0.03877757862210274, -0.02332380972802639, -0.03115187957882881, 0.08677797019481659, 0.028858602046966553, 0.09145060926675797, -0.012553032487630844, 0.07246575504541397, -0.06274964660406113, 0.037177860736846924, 0.0226147398352623, -0.01741017960011959, 0.029221603646874428, 0.0005846760468557477, -0.0820252075791359, -0.07640764862298965, 0.09954116493463516, -0.05204181373119354, -0.08840569108724594, -0.022213447839021683, -0.08701308071613312, -0.03923098370432854, -0.07706428319215775, 0.03270617499947548, 0.030455457046628, 0.0022725036833435297, 0.03877577558159828, -0.09242253750562668, 0.020388904958963394, -0.06306009739637375, -0.07062531262636185, -0.031546760350465775, -0.03371599316596985, 0.03426560387015343, 0.045298267155885696, 0.05433965474367142, -0.0748462826013565, 0.013300818391144276, -0.008453219197690487, 0.056876689195632935, -0.071536123752594, 0.005248079542070627, -0.03639381006360054, -0.023129332810640335, 0.010828484781086445, 0.09592036157846451, -0.02691495046019554, -0.09307652711868286, 0.003190393093973398, -0.07412519305944443, -0.06253834813833237, -0.03361537680029869, -0.028060555458068848, -0.03895818814635277, -0.022883497178554535, 0.05403662845492363, -0.06173279508948326, -0.039142608642578125, -0.01013611163944006, -0.0349278561770916, 0.05566859990358353, -0.07026873528957367, -0.02345428243279457, -0.0015722711104899645, -0.04771893471479416, -0.0420544259250164, -0.07945061475038528, -0.04860761761665344, 0.025618452578783035, -0.06541751325130463, -0.037502776831388474, -0.0389198400080204, -0.03920906037092209, 0.0091232405975461, 0.05576622486114502, -0.060102127492427826, 0.032866884022951126, 0.04631076380610466, -0.046918805688619614, -0.004229368641972542, -0.06008611619472504, 0.009982275776565075, 0.03814368695020676, 0.03912101686000824, -0.08677789568901062, -0.03409944474697113, 0.10236305743455887, 0.004871276672929525, -0.08725675195455551, 0.026630695909261703, 0.01720784604549408, -0.08174529671669006, 0.053550537675619125, -0.013807587325572968, 0.07523785531520844, 0.012616248801350594, -0.021031880751252174, -0.0527472123503685, 0.06593964248895645, 0.050514183938503265, -0.004032263066619635, -0.011401685886085033, 0.0025479719042778015, -0.04209371656179428, -0.0003215898759663105, 0.05104774236679077, 0.04990788549184799, 0.012645300477743149, -0.05336930602788925, -0.049174193292856216, 0.060272131115198135, 0.08860409259796143, 0.04935196042060852, -0.05672292038798332, -0.02956034243106842, -0.0666753277182579, 0.008205852471292019, -0.0643097385764122, 0.034662533551454544, -0.01612257957458496, -0.0702311173081398, -0.02038113959133625, -0.08505275845527649, 0.02064601518213749, 0.02984982542693615, 0.019232111051678658, 0.06611321866512299, -0.04002868011593819, -0.06026611849665642, -0.06415128707885742, -0.08450803905725479, -0.06543880701065063, 0.029548050835728645, 0.01690562255680561, 0.014310382306575775, 0.025903893634676933, -0.046458952128887177, -0.00204130494967103, -0.017334692180156708, -0.017980825155973434, -0.0704093724489212, 0.01080639660358429, -0.05665718391537666, -0.004387800116091967, -0.005291200242936611, -0.003937426954507828, 0.04924842715263367, -0.07069836556911469, -0.08873364329338074, -0.03274339437484741, 0.042106300592422485, 0.007468339055776596, -0.022708429023623466, 0.08068564534187317, -0.11098944395780563, -0.009707284159958363, 0.026941942051053047, -0.04796867072582245, -0.03671332076191902, -0.013375196605920792, 0.008323224261403084, 0.07456998527050018, 0.065411277115345, -0.07870442420244217, -0.06543699651956558, 0.012625218369066715, -0.023194996640086174, 0.019968310371041298, -0.08940701931715012, 0.09148237109184265, 0.03192498907446861, 0.004256944637745619, -0.08479361236095428, 0.039618875831365585, 0.027523376047611237, 0.10739744454622269, -0.07137419283390045, -0.008010662160813808, -0.0007550684968009591, -0.02529609017074108, -0.05941068008542061, 0.053802065551280975, 0.01425077859312296, -0.0450579933822155, 0.0014158929698169231, -0.08221324533224106, -0.031601063907146454, -0.007075417786836624, -0.0700678825378418, 0.08422471582889557, 0.06997281312942505, -0.06990242749452591, -0.096765898168087, -0.013581477105617523, -0.09016242623329163, 0.013353848829865456, 0.10862012952566147, -0.015129689127206802, -0.05559711903333664, -0.0016822903417050838, 0.026430858299136162, 0.04952158033847809, -0.0976424366235733, 0.06458014249801636, 0.023715423420071602, 0.02511521428823471, -0.0037989963311702013, -0.08569560199975967, -0.05322616547346115, -0.05495648458600044, -0.09148307889699936, 9.59251046879217e-05, -0.06899579614400864, 0.06075465306639671, -0.05850495025515556, -0.10029250383377075, -0.07723041623830795, -0.021839987486600876, 0.04836764186620712, 0.06281685084104538, -0.04515478014945984, -0.03396984934806824, 0.006020030938088894, 0.025506697595119476, -0.04710197076201439, 0.05022517591714859, 0.06559084355831146, 0.08021584898233414, 0.04755108803510666, -0.012752305716276169, -0.02484346553683281, -0.04473090544342995, -0.04276074469089508, -0.007668943610042334, -0.07393940538167953, -0.05556396767497063, -0.008512459695339203, -0.04754503443837166, 0.0228312648832798, -0.032014988362789154, 0.04650343582034111, 0.08993474394083023, 0.05202508717775345, -0.00014027144061401486, -0.0034031765535473824, -0.05215742811560631, 0.11572758108377457, -0.07967419177293777, 0.06793558597564697, 0.03333372250199318, -0.046373117715120316, -0.020261036232113838, -0.07775615155696869, -0.08245894312858582, -0.008838707581162453, 0.0790843665599823, 0.07070109248161316, -0.09719433635473251, -0.046324364840984344, -0.023820925503969193, 0.01636331155896187, 0.05389828607439995, -0.03427697345614433, 0.03295983001589775, 0.04088588431477547, 0.021469254046678543, 0.03652416914701462, -0.09366563707590103, 0.033404674381017685, -0.03692278265953064, 0.08715115487575531, -0.029897049069404602, 0.04439070448279381, -0.09088370203971863, -0.03971794620156288, -0.07453658431768417, -0.047628674656152725, 0.015250168740749359, -0.010125181637704372, 0.08442948758602142, -0.0337100587785244, 0.021671492606401443, 0.05514976382255554, -0.07315906137228012, 0.09099875390529633, 0.0072059957310557365, 0.09470948576927185, -0.029918376356363297, -0.016049420461058617, -0.058391913771629333, 0.029341785237193108, -0.037892211228609085, -0.08423683047294617, 0.05806145817041397, -0.06687517464160919, -0.014475535601377487, -0.014914698898792267, 0.04352725297212601, 0.040093984454870224, -0.08400814235210419, 0.11354866623878479, -0.05008506402373314, 0.05823865532875061, 0.056730542331933975, 0.09794602543115616, 0.0024447785690426826, -0.05275536701083183, 0.02371804788708687, 0.015385951846837997, -0.017466764897108078, -0.014284727163612843, -0.07661605626344681, 0.07876951992511749, 0.06843889504671097, -0.01963440701365471, 0.04042308032512665, 0.048479337245225906, -0.012635629624128342, -0.08686474710702896, -0.06107470393180847, 0.10614539682865143, 0.019776389002799988, -0.05364977940917015, -0.007235015742480755, -0.05343547835946083, -0.0886189341545105, 0.09795158356428146, -0.03852704539895058, 0.07254045456647873, -0.007118573412299156, 0.0097877262160182, 0.010606777854263783, 0.0053640129044651985, 0.0778481662273407, 0.007701897528022528, -0.02105133980512619, 0.0597628615796566, 0.09967256337404251, 0.045736901462078094, -0.023640641942620277, 0.0032233267556875944, 0.04747012257575989, -0.035884592682123184, 0.06733771413564682, -0.08245325833559036, -0.02726026624441147, -0.08228664100170135, -0.02978970855474472, -0.0677831694483757, -0.05391937121748924, -0.00843531358987093, 0.026102954521775246, 0.01492107193917036, 0.019321734085679054, 0.03434096649289131, -0.00517842685803771, 0.021929973736405373, 0.06709186732769012, -0.06572741270065308, -0.07323788851499557, 0.05582011491060257, -0.07104016840457916, 0.08348435163497925, 0.08920903503894806, 0.018393779173493385, 0.09288628399372101, -0.028959328308701515, -0.02130901627242565, -0.02295241877436638, -0.07156835496425629, 0.021258721128106117, 0.044204290956258774, -0.07360854744911194, 0.04134979471564293, 0.03324205055832863, 0.042277511209249496, 0.06968320161104202, -0.0350225530564785, -0.05969186872243881, 0.03798181563615799, 0.04042394459247589, 0.0042268093675374985, 0.021472422406077385, 0.009521480649709702, -0.024980882182717323, -0.04889239743351936, 0.025240929797291756, 0.09716540575027466, -0.0335591658949852, 0.08346138149499893, 0.058319393545389175, 0.04207079857587814, -0.0578283928334713, -0.06660592555999756, -0.1059388816356659, 0.04939206317067146, 0.011033562012016773, -0.03654731810092926, -0.10417264699935913, 0.008637083694338799, 0.027517814189195633, 0.004850813653320074, 0.001349512836895883, 0.06591811776161194, -0.030032962560653687, 0.044994525611400604, 0.012005853466689587, -0.04668085277080536, -0.05794830992817879, -0.009269772097468376, -0.0029445600230246782, 0.06425530463457108, -0.002786807017400861, 0.038675978779792786, -0.07081753760576248, 0.05914521962404251, 0.016428373754024506, -0.08560719341039658, 0.052884697914123535, 0.03802528232336044, 0.015148522332310677, -0.09421642124652863, 0.047876112163066864, -0.03630133718252182, 0.019965196028351784, -0.11222144216299057, 0.0005519979167729616, 0.068081796169281, -0.016853585839271545, 0.009853778406977654, -0.00173326232470572, -0.10497691482305527, 0.05126078799366951, -0.06261174380779266, -0.07473809272050858, -0.03769148513674736, -8.222958422265947e-05, 0.048234596848487854, 0.07359448075294495, 0.02350788563489914, -0.023018386214971542, -0.02669634483754635, -0.025498870760202408, 0.032048966735601425, 0.042161911725997925, -0.02846403606235981, -0.02168603613972664, 0.08790119737386703, 0.04916790872812271, -0.025078918784856796, 0.008686250075697899, -0.032462239265441895, 0.057173680514097214, -0.05184381082653999, -0.027099335566163063, -0.020576754584908485, -0.04257505387067795, -0.006547590252012014, -0.04725388064980507, -0.07669001817703247, -0.021887430921196938, 0.006890797056257725, 0.1030956283211708, -0.01716540940105915, 0.04426834359765053, 0.015471721068024635, -0.019975727424025536, 0.0013498771004378796, 0.01217711716890335, -0.07236184179782867, -0.043697524815797806, -0.04279666393995285, 0.06470823287963867, -0.04952617362141609, 0.03807826712727547, 0.07074202597141266, -0.07367952167987823, -0.02019876055419445, 0.012992257252335548, 0.06808123737573624, -0.01915048435330391, -0.08547686040401459, -0.04263562336564064, 0.006424145773053169, -0.03085729107260704, -0.029825927689671516, 0.11213620752096176, 0.08467426151037216, 0.07524817436933517, -0.08038916438817978, -0.04484820365905762, 0.01444205641746521, -0.05987349897623062, -0.11036829650402069, 0.09639301896095276, 0.04733824357390404, -0.03898654878139496, -0.04899473860859871, -0.10765154659748077, -0.02310745231807232, 0.0699271410703659, 0.059092774987220764, -0.024538451805710793, 0.03861815109848976, -0.027032772079110146, 0.043016217648983, -0.11498581618070602, 0.09865328669548035, 0.052105583250522614, -0.0309166107326746, -0.05625927075743675, 0.0978451669216156, 0.08485328406095505, 0.03723069280385971, -0.05625966563820839, 0.048742812126874924, -0.061574775725603104, -0.08575629442930222, 0.0733889490365982, 0.0023447037674486637, -0.01995791122317314, -0.11331179738044739, 0.023525448516011238, 0.03813076391816139, -0.12211837619543076, -0.04582594335079193, -0.04580859839916229, -0.011194703169167042, 0.050248537212610245, 0.04486820846796036, -0.01216848660260439, 0.09715843200683594, -0.011970256455242634, 0.04344204068183899, -0.023370344191789627, -0.08849859982728958, -0.0026744354981929064, 0.0175991952419281, -0.06541240215301514, -0.06527549028396606, -0.0661882683634758, 0.028598906472325325, 0.07890984416007996, -0.09605269879102707, 0.057821787893772125, 0.012156725861132145, -0.05090443789958954, 0.031631723046302795, -0.04418495297431946, -0.09557908028364182, -0.07787945121526718, 0.0424368754029274, 0.04000093415379524, -0.025012720376253128, -0.0458095483481884, 0.01686996780335903, 0.013747939839959145, -0.043048687279224396, -0.03959732875227928, 0.06081492453813553, 0.05964153632521629, -0.028787117451429367, 0.008344979956746101, -0.09694184362888336, -0.06839501857757568, 0.04285256937146187, 0.0031525695230811834, 0.03285963088274002, -0.05461877956986427, -0.029671551659703255, 0.06564033031463623, 0.0446588434278965, -0.0390503890812397, -0.009464103728532791, -0.035720355808734894, -0.0007524769753217697, -0.04676947742700577, -0.02750237286090851, 0.09511619061231613, -0.026047248393297195, 0.04328450188040733, 0.06646756082773209, 0.005046096630394459, -0.08935948461294174, -0.02546525001525879, -0.015436345711350441, 0.03926330804824829, 0.05595890060067177, -0.0364530012011528, -0.0685095563530922, 0.06176881864666939, -0.031101182103157043, 0.023268532007932663, -0.04738682135939598, -0.03562377020716667, -0.034662842750549316, -0.05844026431441307, 0.05281594395637512, -0.033736616373062134, -0.04377833008766174, -0.006063796579837799, -0.029134370386600494, -0.07810120284557343, -0.016478780657052994, -0.011441103182733059, 0.009478043764829636, -0.011606387794017792, -0.03759828954935074, -0.09798778593540192, -0.09113481640815735, 0.03338579088449478, -0.03679145872592926, 0.030328908935189247, 0.08744233101606369, 0.026896214112639427, -0.0010192238260060549, 0.05304263159632683, -0.061878737062215805, 0.057308394461870193, 0.08145786821842194, 0.00978431198745966, 0.049700118601322174, 0.0012699449434876442, 0.03146602585911751, -0.01918105036020279, -0.09106064587831497, -0.035614125430583954, -0.07685837149620056, -0.09399180114269257, -0.06540987640619278, -0.007245148066431284, 0.06698989123106003, -0.04138174653053284, -0.03631279245018959, -0.08329854160547256, 0.07811029255390167, 0.017256446182727814, -0.05585944652557373, -0.09186212718486786, 0.05533171072602272, -0.02472352236509323, 0.014069016091525555, -0.053630243986845016, 0.011681107804179192, -0.05193360894918442, 0.026413261890411377, 0.04141763225197792, -0.0013146842829883099, 0.03346043825149536, -0.06236371025443077, 0.042757753282785416, 0.028482912108302116, -0.03993011265993118, 0.06338229775428772, -0.0738779827952385, -0.04788960888981819, -0.012460459023714066, 0.08629828691482544, -0.000733796099666506, 0.07352440059185028, 0.043199192732572556, -0.03508101776242256, -0.0049050613306462765, 0.052573081105947495, -0.04519146308302879, 0.004401498008519411, 0.05048248916864395, -0.05075083673000336, -0.07103631645441055, 0.018001984804868698, 0.012149443849921227, 0.012083740904927254, 0.05080806836485863, -0.08520642668008804, 0.05100957676768303, 0.06656423956155777, 0.06195894256234169, -0.027896232903003693, -0.09006235003471375, -0.006894255522638559, -0.03822784870862961, 0.05081440135836601, 0.003232609946280718, 0.05977131426334381, -0.005225813947618008, 0.016079790890216827, -0.03594024106860161, 0.0018623051000759006, 0.07566697150468826, 0.015641212463378906, -0.016697928309440613, 0.03812666982412338, -0.03884468600153923, 0.04078768938779831, 0.058555569499731064, -0.046634234488010406, -0.0015861530555412173, 0.07463838905096054, -0.07078449428081512, 0.047596897929906845, -0.05362200364470482, 0.082015760242939, 0.041899148374795914, 0.03370055928826332, -0.009580787271261215, 0.027699235826730728, -0.045974016189575195, -0.09017950296401978, 0.04899347573518753, -0.07404903322458267, -0.020465293899178505, -0.037015076726675034, 0.040512580424547195, -0.0824618712067604, 0.006669335067272186, 0.05795521289110184, 0.04927820712327957, -0.06626470386981964, -0.08295714110136032, -0.009603949263691902, 0.0844961404800415, 0.08968646079301834, -0.04827432334423065, -0.06434113532304764, 0.09043966233730316, 0.02388329803943634, -0.07366123795509338, 0.05950583145022392, -0.037232425063848495, 0.017410488799214363, 0.050673700869083405, -0.02171475626528263, -0.06044982373714447, -0.03228571638464928, 0.007191333454102278, -0.03830474987626076, -0.002124500460922718, 0.04115510359406471, -0.044752415269613266, 0.06049289554357529, -0.0525180883705616, 0.07065404951572418, -0.05659383162856102, -0.0662735104560852, -0.07834537327289581, -0.09827098250389099, -0.04642660543322563, 0.024646813049912453, 0.054597288370132446, -0.06648799031972885, -0.009262529201805592, 0.059868983924388885, -0.03484655171632767, -0.07876073569059372, -0.07936260104179382, -0.06104779988527298, 0.0004886203678324819, -0.06338770687580109, 0.0024385987780988216, -0.047715283930301666, -0.005274435970932245, -0.03824322670698166, -0.07407781481742859, -0.07396329194307327, 0.03808658570051193, 0.06904446333646774, 0.08515344560146332, 0.017462093383073807, 0.007913529872894287, 0.06298311054706573, -0.011793484911322594, -0.05105297639966011, -0.08755306154489517, 0.06159062683582306, 0.04108366370201111, -0.09019200503826141, -0.009465962648391724, -0.041740354150533676, 0.04066553711891174, 0.047908198088407516, -0.023128675296902657, -0.022493597120046616, -0.03660504147410393, -0.03440007194876671, 0.06543972343206406, -0.0985712856054306, 0.06052234396338463, 0.05589193478226662, -0.08625385165214539, 0.062194135040044785, -0.04462531954050064, 0.05237729847431183, 0.020716119557619095, -0.01657162979245186, -0.03762108087539673, -0.08259687572717667, 0.0338335745036602, 0.005276148207485676, 0.042668819427490234, 0.09818978607654572, -0.0947827622294426, 0.02657746709883213, -0.0013200307730585337, 0.0183180533349514, -0.05079676955938339, -0.018773153424263, -0.059997398406267166, -0.10583493858575821, -0.027335256338119507, -0.10700402408838272, 0.020863022655248642, -0.07569871842861176, -0.0608159676194191, -0.05072667449712753, -0.026710055768489838, -0.01720971055328846, -0.05576741322875023, -0.014202810823917389, 0.07759113609790802, 0.008410296402871609, -0.03967811167240143, -0.014858587644994259, 0.011025456711649895, -0.05673231557011604, 0.09460017085075378, 0.05805971473455429, -0.07217124849557877, -0.06425752490758896, 0.018492011353373528, -0.001842212164774537, 0.07183260470628738, -0.0866110697388649, 0.07990197837352753, -0.0019523936789482832, 3.994863072875887e-05, 0.09134501218795776, -0.053719308227300644, 0.03783966973423958, -0.018872927874326706, 0.017522605136036873, 0.05792298913002014, -0.018553756177425385, -0.06060101464390755, 0.03519240394234657, 0.0506131611764431, 0.015791377052664757, -0.048417821526527405, -0.01909731887280941, -0.09744934737682343, -0.006127988453954458, -0.03707558289170265, 0.02044774778187275, 0.0661868005990982, 0.03897923603653908, -0.07444711774587631, -0.034670647233724594, -0.0817011147737503, 0.011478852480649948, 0.018155904486775398, 0.01720757968723774, -0.006878416519612074, 0.03366710618138313, -0.04054905101656914, -0.06271872669458389, 0.03456221520900726, 0.026327725499868393, -0.033297013491392136, -0.014813251793384552, 0.03466251492500305, 0.040523990988731384, -0.010339430533349514, -0.06369275599718094, -0.04312955215573311, 0.008571716025471687, -0.002060492290183902, 0.03309614956378937, -0.13021716475486755, 0.05561825633049011, 0.00857964064925909, 0.07473721355199814, -0.002585482317954302, -0.047901831567287445, 0.0653529167175293, -0.0942193865776062, -0.01592765375971794, -0.0070193312130868435, -0.09323963522911072, -0.004946762230247259, 0.04180850461125374, -0.06400806456804276, 0.018201682716608047, -0.0266133900731802, 0.03589174896478653, -0.05108577758073807, -0.04801063612103462, -0.11005629599094391, -0.010160324163734913, 0.027655089274048805, -0.021686693653464317, 0.04985922574996948, -0.05107162892818451, 0.043445147573947906, -0.11353668570518494, 0.0241230558604002, 0.03813992440700531, -0.09238822758197784, 0.0329270102083683, -0.09646742045879364, -0.053153157234191895, -0.0114017678424716, 0.04662969335913658, 0.05839335173368454, -0.008465063758194447, 0.00496989581733942, -0.008301273919641972, 0.048041779547929764, 0.034052588045597076, 0.046136703342199326, 0.04919463396072388, -0.006192959379404783, -0.07055459171533585, 0.012734001502394676, 0.04695386812090874, -0.028561577200889587, -0.0789753794670105, -0.0657849907875061, 0.009976062923669815, -0.044318463653326035, -0.04321351647377014, 0.02436731569468975, 0.05033963918685913, 0.04580838233232498, 0.04659342020750046, 0.06615269184112549, -0.00974892545491457, -0.06804603338241577, 0.08323360234498978, -0.028140591457486153, -0.05091078206896782, 0.014154378324747086, 0.013923355378210545, -0.026118595153093338, 0.019719261676073074, 0.05489933863282204, -0.04698881506919861, -0.009982987307012081, -0.03914602845907211, -0.008914176374673843, 0.05620047450065613, -0.05541199818253517, 0.07089609652757645, 0.0670921579003334, -0.06682918220758438, 0.03647675737738609, -0.03599962219595909, -0.001320268609561026, 0.02486424334347248, -0.0700829029083252, -0.10123661905527115, 0.03036223165690899, -0.041334498673677444, 0.04593268781900406, -0.007157906889915466, 0.00873046275228262, 0.03294387832283974, 0.00821594800800085, -0.028282731771469116, -0.0313953198492527, -0.007388253230601549, -0.0012190251145511866, -0.050387680530548096, -0.010317999869585037, -0.04594989866018295, 0.04339050129055977, 0.08706101030111313, 0.04763214290142059, 0.03426491841673851, 0.0050930664874613285, 0.0067803263664245605, 0.013684342615306377, 0.07265976071357727, -0.05790703371167183, -0.011447974480688572, 0.01659161038696766, 0.003039915580302477, 0.041757483035326004, 0.02210860513150692, 0.07016240060329437, -0.10680541396141052, -0.05723615363240242, 0.07260426878929138, -0.01754375733435154, -0.052129652351140976, -0.07134445756673813, 0.06427159905433655, -0.02155580185353756, -0.029546041041612625, -0.015369155444204807, 0.025244371965527534, -0.02492489293217659, 0.05546610429883003, -0.021675998345017433, 0.018511878326535225, -0.01742200367152691, 0.034875135868787766, 0.05341124162077904, -0.005192763637751341, 0.03377632051706314, 0.00877840630710125, -0.0045384615659713745, -0.04146154969930649, 0.036315854638814926, 0.08737734705209732, -0.09100531786680222, 0.02176293358206749, 0.022774599492549896, 0.05226018279790878, 0.024420185014605522, -0.05084759742021561, 0.04858386889100075, -0.05225808545947075, 0.01434763427823782, 0.023569194599986076, -0.04338010773062706, 0.0027876433450728655, -0.09117183089256287, -0.09130477160215378, -0.029301617294549942, -0.06599252671003342, -0.0800970271229744, 0.05677188187837601, -0.08549495041370392, -0.04059378802776337, -0.09414032846689224, -0.06300073862075806, 0.07449183613061905, -0.004306995775550604, 0.017829308286309242, -0.08096949756145477, 0.01168267335742712, 0.022955095395445824, -0.03790856525301933, 0.050028491765260696, -0.04774847999215126, -0.07427875697612762, -0.04801114648580551, -0.09206120669841766, 0.07833297550678253, -0.02400706522166729, -0.04180051386356354, -0.08818919956684113, 0.00604423088952899, -0.038335662335157394, 0.060558490455150604, 0.0013968388084322214, 0.07544190436601639, 0.04322930425405502, 0.024214256554841995, -0.05890773981809616, -0.09747131168842316, -0.020150115713477135, -0.017344634979963303, -0.031541235744953156, 0.016665101051330566, -0.010596740059554577, 0.023007182404398918, 0.06338626891374588, -0.018969841301441193, 0.0651513859629631, -0.0057501643896102905, -0.0074372440576553345, 0.061882052570581436, 0.07448498904705048, -0.07537655532360077, 0.046859435737133026, 0.08607222884893417, 0.004913097247481346, -0.08235906809568405, 0.04710562527179718, -0.06070498377084732, 0.03256819769740105, 0.058375682681798935, 0.0474347360432148, 0.05208756774663925, -0.0485256165266037, -0.009845095686614513, -0.045586083084344864, -0.05124398320913315, 0.025978313758969307, 0.0855816975235939, 0.007949813269078732, -0.014342421665787697, -0.06079642102122307, 0.012596284039318562, -0.02678309567272663, -0.06787554919719696, 0.014005451463162899, 0.06304461508989334, -0.04646945744752884, -0.027116842567920685, -0.08536449074745178, 0.007454717997461557, 0.07541344314813614, -0.09398327767848969, -0.01126833911985159, -0.01352021936327219, 0.039648622274398804, 0.014680118300020695, 0.02573779597878456, 0.043070707470178604, -0.04859347641468048, -0.10886934399604797, 0.03472249582409859, -0.06672997772693634, 0.0780877098441124, -0.01699148304760456, -0.049396295100450516, -0.03114405833184719, -0.014718052931129932, -0.06652914732694626, 0.0613396130502224, -0.04035160690546036, -0.04756981134414673, 0.04045013338327408, 0.008286652155220509, -0.019704461097717285, 0.019999556243419647, -0.014386882074177265, 0.07323019951581955, -0.06410946696996689, -0.09559612721204758, -0.060561951249837875, -0.011738749220967293, 0.019140897318720818, 0.014096944592893124, 0.029129285365343094, -0.07965350151062012, -0.0850367471575737, 0.013553790748119354, -0.03422419726848602, 0.060511402785778046, -0.019654743373394012, 0.01298781018704176, -0.0578167699277401, -0.03775804489850998, -0.011976253241300583, 0.036784153431653976, -0.024379858747124672, -0.03952780365943909, 0.0003962484188377857, -0.056507643312215805, 0.07163039594888687, -0.018358461558818817, 0.06893642991781235, 0.0188894160091877, -0.03479403629899025, -0.08312412351369858, -0.01613585464656353, 0.0004994178307242692, 0.005728056188672781, 0.05724859982728958, 0.04986976459622383, -0.07499033957719803, 0.05467889830470085, -0.06762447953224182, 0.05834782496094704, -0.01970336213707924, -0.05387745052576065, 0.021904369816184044, 0.05861514434218407, 0.07231906056404114, -0.06108974665403366, 0.0836082473397255, 0.020060619339346886, -0.022232796996831894, 0.028351016342639923, 0.033836159855127335, 0.03980618715286255, -0.032679226249456406, 0.08244289457798004, -0.060873355716466904, 0.04329729452729225, 0.05458279326558113, -0.06490867584943771, 0.06392320990562439, 0.02994777448475361, -0.07460562139749527, -0.010270842351019382, -0.06889714300632477, 0.06614729017019272, 0.04682955518364906, -0.01263659168034792, 0.07208866626024246, -0.07189378887414932, -0.0647512823343277, -0.026226665824651718, 0.05521024763584137, -0.0745977908372879, -0.0065571158193051815, 0.013266629539430141, 0.08407381176948547, -0.008813910186290741, -0.03950444236397743, -0.01315377652645111, -0.08580417931079865, 0.07254645228385925, -0.08585099130868912, 0.05433196946978569, 0.04679259657859802, -0.07604168355464935, 0.000587814545724541, 0.03426807373762131, 0.05185936018824577, 0.035915445536375046, -0.06462842226028442, -0.00321757304482162, -0.022196101024746895, 0.014263750985264778, 0.049735818058252335, 0.059404753148555756, -0.08305878192186356, 0.05301700532436371, 0.0724172443151474, 0.02062067575752735, 0.07997764647006989, 0.037221044301986694, 0.08927065134048462, 0.021312065422534943, 0.0028663710691034794, 0.07519685477018356, 0.03445611894130707, -0.08207140117883682, -0.06565742939710617, -0.030196191743016243, 0.014311163686215878, 0.030905837193131447, -0.02433098293840885, 0.03684855252504349, 0.07361461967229843, -0.04420778527855873, 0.06474349647760391, 0.02755315601825714, -0.009197216480970383, 0.036862701177597046, -0.03792060539126396, -0.08371121436357498, 0.01666981913149357, -0.028870103880763054, -0.07881656289100647, -0.035302113741636276, -0.06102468818426132, -0.013964926823973656, -0.04032720997929573, 0.027560759335756302, 0.011631990782916546, 0.020624330267310143, 0.008555049076676369, 0.0066401418298482895, 0.023222599178552628, 0.08318904787302017, 0.026667678728699684, -0.06332390010356903, -0.04031691327691078, 0.03427840769290924, 0.024844268336892128, -0.013231121003627777, 0.044640008360147476, 0.04990560561418533, -0.07488510757684708, 0.05420515313744545, -0.016922052949666977, 0.016784515231847763, -0.002537722699344158, -0.031728532165288925, -0.026965128257870674, -0.017845144495368004, -0.04205697029829025, -0.004294962622225285, -0.011753072030842304, 0.04421075060963631, -0.004297191277146339, -0.024905722588300705, 0.03946895897388458, -0.08913225680589676, -0.0720532238483429, -0.0014228018699213862, 0.07593391835689545, -0.06655442714691162, 0.004441572818905115, -0.026714779436588287, 0.060892701148986816, -0.015730060636997223, 0.062372010201215744, -0.047263264656066895, -0.002180071547627449, 0.05578373000025749, -0.015329472720623016, 0.07067635655403137, 0.032590169459581375, -0.012144888751208782, -0.07211428880691528, 0.0338178388774395, -0.012796015478670597, 0.012622282840311527, 0.02245408296585083, 0.00018898963753599674, -0.07524552941322327, 0.09072930365800858, 0.06680314987897873, -0.07386188209056854, -0.02904498018324375, 0.03600012883543968, -0.0015419271076098084, -0.010086126625537872, -0.022611381486058235, 0.08285557478666306, 0.05248429253697395, -0.05773768946528435, 0.05456748604774475, 0.0603751614689827, -0.06419826298952103, 0.04252959042787552, 0.07950206845998764, 0.027446316555142403, 0.043511878699064255, 0.10788892209529877, 0.08722664415836334, -0.05616801977157593, 0.0483328178524971, -0.017820853739976883, -0.015987476333975792, 0.03709934651851654, -0.020451266318559647, 0.058857712894678116, -0.061543338000774384, -0.07625386863946915, 0.027924826368689537, -0.0980030968785286, 0.015588425099849701, 0.023680860176682472, 0.012851795181632042, -0.016824031248688698, -0.06775592267513275, 0.06938018649816513, -0.07268251478672028, 0.030577419325709343, -0.0064732893370091915, 0.03316151723265648, 0.008372203446924686, 0.04176211357116699, 0.024046769365668297, -0.0024132095277309418, -0.04844790697097778, 0.07215388864278793, 0.005626116879284382, -0.027029190212488174, 0.03884311765432358, 0.07278041541576385, -0.03163790702819824, -0.10209144651889801, -0.1012948602437973, -0.018257958814501762, 0.0710657611489296, 0.04234963282942772, -0.06407278776168823, 0.030609065666794777, 0.009046007879078388, 0.10060799866914749, 0.052789945155382156, -0.08503822237253189, -0.0506274588406086, 0.01533557940274477, -0.05116972327232361, -0.028856754302978516, 0.007983803749084473, -0.036225076764822006, -0.0006207283586263657, 0.029898911714553833, -0.03412923216819763, 0.06663454324007034, -0.042588766664266586, 0.05634090304374695, 0.026005899533629417, -0.04514436796307564, -0.06824152916669846, 0.05100718513131142, 0.10846710950136185, 0.04648697376251221, -0.09681887924671173, 0.05426102876663208, 0.0551801472902298, 0.045963872224092484, 0.06679610908031464, -0.06263700872659683, 0.09392835199832916, 0.02102070115506649, 0.0035157641395926476, 0.029823241755366325, -0.0470561720430851, 0.021274445578455925, -0.06715042144060135, -0.044443730264902115, 0.022532088682055473, 0.05681896209716797, -0.04758131876587868, 0.0930878072977066, 0.045890070497989655, -0.04230612888932228, -0.03526640683412552, -0.1335602104663849, 0.03655623272061348, 0.007599310018122196, 0.06541723012924194, -0.05537029355764389, 0.017011733725667, 0.08415601402521133, -0.044302791357040405, 0.02273768186569214, 0.019173141568899155, 0.029040107503533363, -0.028289884328842163, -0.004743835888803005, 0.027550822123885155, 0.10225971788167953, -0.03838983178138733, -0.05127835273742676, 0.052126094698905945, 0.031035415828227997, -0.09334932267665863, -0.02620883285999298, -0.05104193463921547, 0.036365337669849396, -0.020244119688868523, -0.014064921997487545, 0.05431990697979927, -0.04920116811990738, 0.0009556278237141669, -0.09965020418167114, 0.008904768154025078, -0.037684693932533264, 0.0334579199552536, -0.1005689799785614, 0.03614400327205658, 0.010868865996599197, -0.08627079427242279, -0.025962339714169502, 0.008261709474027157, 0.012080643326044083, -0.09998853504657745, -0.06284300237894058, -0.04137502610683441, -0.0018421857384964824, 0.029131917282938957, -0.03630610555410385, -0.07313162833452225, 0.02227914147078991, 0.05387183651328087, -0.10385319590568542, 0.07999622821807861, -0.06007299944758415, 0.05253448337316513, -0.033696673810482025, -0.06071829795837402, -0.08310963213443756, -0.03769068047404289, 0.0374314971268177, 0.03527568653225899, -0.004421119578182697, 0.05567564070224762, -0.014795913361012936, 0.009147821925580502, 0.03585444763302803, 0.027963025495409966, 0.0891738086938858, 0.032318100333213806, 0.01242799498140812, 0.0719490572810173, 0.005970449186861515, -0.08169501274824142, 0.012271515093743801, 0.05195105820894241, -0.04408475011587143, -0.129958376288414, 0.022400854155421257, -0.03406825661659241, -0.07610597461462021, 0.07735550403594971, 0.0023712830152362585, 0.050207093358039856, 0.043971046805381775, -0.024131538346409798, 0.07417625188827515, 0.03892102465033531, -0.06357866525650024, 0.006407196167856455, 0.05497141182422638, 0.052494943141937256, -0.0764409527182579, -0.0041447291150689125, -0.02714729867875576, -0.017254211008548737, 0.034551866352558136, 0.07700315117835999, -0.03104522079229355, 0.026977822184562683, -0.044170208275318146, 0.04520759359002113, 0.034254398196935654, -0.09907730668783188, -0.07303515821695328, -0.06468681246042252, -0.041151490062475204, -0.04558302089571953, -0.08344417810440063, -0.04015074670314789, 0.07279710471630096, 0.020937133580446243, -0.024845391511917114, 0.010720927268266678, 0.03949392959475517, 0.045990895479917526, 0.03012743778526783, -0.07400091737508774, -0.03602954372763634, 0.07648498564958572, -0.0016817179275676608, -0.027872668579220772, -0.056488584727048874, 0.0764886811375618, -0.04359954223036766, -0.07323896139860153, -0.055173736065626144, -0.024293309077620506, -0.01862671598792076, -0.08365961909294128, 0.08823812007904053, 0.02310006506741047, 0.015822848305106163, -0.08800097554922104, -0.06618613749742508, -0.008862944319844246, 0.08488035202026367, -0.005508371163159609, -0.06298556178808212, 0.048349108546972275, -0.026112150400877, -0.07308590412139893, -0.04700832441449165, 0.017287541180849075, 0.04197388142347336, -0.025941934436559677, 0.054706770926713943, -0.07125092297792435, -0.043284837156534195, -0.023802099749445915, 0.008528637699782848, 0.06454534828662872, -0.019508417695760727, -0.017333321273326874, -0.039970237761735916, 0.018324637785553932, -0.015600577928125858, -0.0008472824702039361, 0.04103454574942589, -0.05290622636675835, -0.06106624752283096, -0.06015089154243469, -0.0007019665790721774, 0.02519291639328003, -0.03759126737713814, -0.01786654070019722, 0.04999283701181412, -0.03776567056775093, -0.0347868837416172, -0.0448266826570034, 0.011447209864854813, -0.03082548826932907, -0.05368153750896454, -0.0956377163529396, -0.04694267362356186, 0.0035907989367842674, -0.045547965914011, -0.00042660842882469296, 0.055955301970243454, 0.0257234089076519, -0.030796337872743607, 0.002828564727678895, -0.035906124860048294, -0.0617624968290329, 0.06385292857885361, 0.07173437625169754, -0.06486858427524567, -0.04282860830426216, -0.009447290562093258, 0.0902615338563919, -0.027828991413116455, 0.04731694981455803, -0.05707500874996185, -0.002566679148003459, -0.06500241905450821, 0.034722961485385895, -0.043329186737537384, -0.03696969151496887, 0.008594969287514687, 0.08552204817533493, 0.015439473092556, 0.03974569961428642, 0.08073882013559341, 0.04921530932188034, 0.08970202505588531, -0.010954889468848705, -0.010595829226076603, -0.09131911396980286, 0.032173287123441696, -0.07386656850576401, -0.0640413910150528, 0.09877356886863708, -0.025262149050831795, -0.019557708874344826, -0.05238084867596626, -0.0248752199113369, 0.07550397515296936, -0.04448377713561058, -0.0076331100426614285, -0.048378027975559235, -0.02935171313583851, 0.03437820076942444, 0.0022625720594078302, 0.03465462848544121, 0.025171471759676933, -0.06536194682121277, 0.06812521070241928, 0.021036647260189056, -0.04023953899741173, -0.07637060433626175, -0.00991459283977747, -0.04454253613948822, -0.05725036561489105, 0.01231720857322216, 0.002416895003989339, 0.04278067499399185, 0.014192688278853893, 0.026577550917863846, 0.047303762286901474, -0.04455190524458885, -0.03302662819623947, 0.018895326182246208, -0.030232638120651245, 0.049161508679389954, -0.05280383303761482, 0.014115183614194393, 0.041021667420864105, -0.014836326241493225, 0.04533275216817856, 0.030940797179937363, 0.08626429736614227, 0.06304623931646347, 0.05957859009504318, 0.0748382955789566, -0.050638556480407715, -0.04532280191779137, 0.04019802063703537, 0.048759933561086655, -0.047905854880809784, 0.09264261275529861, -0.022595500573515892, 0.07411611080169678, -0.08028292655944824, 0.07839526236057281, -0.07171688228845596, 0.058204058557748795, -0.04836821183562279, -0.060582466423511505, -0.03428169712424278, -0.029766269028186798, -0.02374710515141487, -0.021936893463134766, -0.044561661779880524, -0.013786189258098602, 0.019162451848387718, 0.03751698136329651, -0.0179281048476696, -0.08680491149425507, -0.012064979411661625, 0.08649831265211105, 0.016554538160562515, 0.07323674857616425, 0.00293913995847106, 0.07242807000875473, 0.06747743487358093, 0.05446624755859375, 0.010437211953103542, 0.029831890016794205, -0.01977912336587906, 0.057531170547008514, 0.004511102568358183, 0.007634633686393499, 0.061293963342905045, -0.09718608856201172, -0.07538039237260818, 0.05876681208610535, -0.026649631559848785, 0.051172204315662384, 0.025054197758436203, 0.01074761152267456, -0.05948972329497337, 0.00619784090667963, -0.029715387150645256, -0.0241645909845829, -0.01757863722741604, 0.006362335290759802, -0.0005503948777914047, -0.017560603097081184, 0.01921292394399643, 0.062187425792217255, 0.026791591197252274, -0.02706102468073368, 0.055731870234012604, 0.011653782799839973, 0.07423572987318039, -0.08513090014457703, 0.04896644875407219, 0.003421859350055456, 0.01982944831252098, 0.03382878378033638, -0.037090808153152466, -0.0523352324962616, -0.023451153188943863, 0.04808215796947479, -0.02166610211133957, 0.08687662333250046, -0.024103743955492973, -0.0249747596681118, -0.05485764890909195, -0.04129047319293022, -0.04447716102004051, 0.019719135016202927, -0.03681427240371704, 0.006347200367599726, 0.061014678329229355, 0.03218185156583786, 0.026509631425142288, -0.0812898576259613, -0.025932032614946365, 0.04074601083993912, 0.062162842601537704, 0.05991989001631737, 0.020791901275515556, 0.0286277886480093, 0.0066655161790549755, -0.007241017650812864, -0.07179145514965057, -0.0038801119662821293, -0.03589414432644844, -0.024203281849622726, 0.042530812323093414, -0.02612251788377762, 0.06419132649898529, -0.03210412338376045, 0.061608847230672836, 0.05216962471604347, -0.07003895938396454, 0.07433970272541046, -0.03346981108188629, 0.04792856052517891, -0.02180851250886917, 0.02721860073506832, -0.02151828445494175, 0.043499790132045746, 0.02114236168563366, -0.1107642874121666, -0.030164316296577454, -0.01864161528646946, -0.08623583614826202, -0.0009369996259920299, -0.038629788905382156, 0.0265412125736475, -0.05969111621379852, 0.05219344049692154, 0.08865363150835037, 0.034543950110673904, -0.030668925493955612, 0.07058648020029068, -0.05665139481425285, -0.020391561090946198, -0.01350065041333437, 0.03312917426228523, 0.04929450899362564, -0.09773276746273041, 0.019431497901678085, -0.0242132730782032, 0.06021856516599655, 0.10147615522146225, -0.031402941793203354, 0.061375025659799576, 0.019663412123918533, 0.06562284380197525, -0.0475018136203289, 0.0700950101017952, 0.02356358803808689, -0.021798646077513695, -0.08158095926046371, 0.002859090454876423, 0.011269226670265198, -0.021595807746052742, 0.023774972185492516, -0.09209958463907242, -0.04269309341907501, -0.09410034865140915, -0.09818967431783676, -0.07941686362028122, -0.07286584377288818, -0.0157438013702631, -0.0975094735622406, 0.009926640428602695, -0.02456626482307911, -0.01390353124588728, -0.059966426342725754, 0.003509932430461049, 0.04632479324936867, -0.07078414410352707, 0.06876125186681747, 0.051071155816316605, 0.0007624668651260436, 0.07271099090576172, 0.06738357245922089, 0.04569458216428757, -0.07726696133613586, -0.003972954582422972, -0.026082735508680344, 0.05655563250184059, -0.027042314410209656, -0.024872181937098503, -0.0038726557977497578, -0.045374080538749695, 0.005800979677587748, 0.013317672535777092, -0.006265426054596901, -0.022660212591290474, -0.049628861248493195, -0.03989530727267265, 0.007251683622598648, -0.0902392566204071, -0.03990752249956131, 0.0927872359752655, 0.05189148336648941, -0.008269932121038437, 0.02006283402442932, -0.02059108018875122, 0.09192675352096558, 0.03773488849401474, -0.0538128986954689, -0.06661570817232132, 0.06635051965713501, -0.04099021106958389, 0.06299465149641037, 0.002360572572797537, -0.10184410214424133, -0.0006616935133934021, 0.009041138924658298, -0.06478564441204071, -0.07737819105386734, 0.057204484939575195, -0.022308697924017906, -0.028857853263616562, 0.04319211468100548, 0.07208307832479477, 0.0966147854924202, -0.043268270790576935, -0.012379813008010387, -0.0028910282999277115, -0.021893547847867012, -0.03118622675538063, 0.05581723526120186, -0.09062916040420532, -0.09202954918146133, 0.09550213813781738, 0.07972097396850586, 0.012331848032772541, 0.039520684629678726, -0.05648336187005043, 0.03166332468390465, -0.05974587798118591, 0.08313088119029999, -0.04501311853528023, 0.09081196784973145, -0.08791348338127136, -0.005909053608775139, 0.013435497879981995, -0.054464925080537796, 0.028525035828351974, -0.08735863119363785, 0.04288485646247864, 0.007129192352294922, -0.037925414741039276, -0.09312374889850616, 0.015750974416732788, -0.013508334755897522, -0.011588008143007755, -0.10054419934749603, 0.04283110797405243, -0.06892383843660355, 0.04571573808789253, -0.053531136363744736, 0.04132210463285446, -0.0005173618556000292, 0.049760933965444565, -0.0026344626676291227, -0.08600127696990967, -0.03385443985462189, 0.03699911758303642, -0.002592473290860653, 0.055323679000139236, 0.04996504634618759, 0.007462258916348219, -0.08559586107730865, -0.011140396818518639, -0.06927431374788284, -0.005301977042108774, 0.07703571021556854, -0.0311379823833704, 0.014749402180314064, 0.02141406387090683, 0.0294950008392334, 0.04268927872180939, 0.026359090581536293, 0.045196957886219025, -0.07599589228630066, -0.004568330477923155, -0.08297618478536606, -0.019400564953684807, 0.06606075167655945, 0.07482870668172836, -0.0742729902267456, -0.02381829544901848, 0.040648724883794785, -0.07799742370843887, -0.11017944663763046, 0.06280583143234253, -0.021168049424886703, 0.06238558143377304, 0.02676958031952381, 0.013091723434627056, 0.019653283059597015, -0.06978820264339447, 0.05984776094555855, -0.027131283655762672, 0.042798735201358795, -0.0010941061191260815, 0.0089713791385293, -0.0036614874843508005, -0.025229481980204582, 0.02932984009385109, 0.010063466615974903, 0.0386258140206337, 0.03060649149119854, -0.021519334986805916, -0.06941475719213486, -0.044740334153175354, -0.05122324451804161, -0.051243893802165985, 0.01950805075466633, 0.06130778416991234, -0.04114958271384239, 0.033816996961832047, -0.05915984883904457, 0.06635459512472153, 0.09392675012350082, 0.05313463509082794, 0.06954742223024368, -0.020179178565740585, 0.03353622928261757, -0.06102956458926201, 0.007062368094921112, -0.0698816254734993, 0.018712449818849564, -0.01524655893445015, -0.041234664618968964, -0.04332972317934036, 0.07919604331254959, 0.06336432695388794, 0.09651830792427063, -0.06963513046503067, -0.029760904610157013, 0.015582328662276268, -0.009796962141990662, -0.0808810293674469, 0.0916927233338356, 0.056615423411130905, 0.034816447645425797, -0.08237823098897934, -0.00861089676618576, 0.06762602925300598, 0.08079531043767929, -0.0752391517162323, 0.02173103578388691, 0.04715016111731529, -0.05822480842471123, 0.043261006474494934, 0.03862522915005684, -0.0009033224196173251, 0.09493905305862427, -0.05639426410198212, 0.04482880234718323, -0.09187234938144684, 0.016020044684410095, 0.025582417845726013, 0.08579675853252411, -0.0452752560377121, 0.0045614768750965595, -0.025957394391298294, -0.05222349241375923, 0.00894784927368164, 0.012593938037753105, 0.09303958714008331, 0.06357067078351974, -0.03090081550180912, 0.006166934501379728, 0.007918191142380238, 0.05298249050974846, -0.03564196452498436, 0.07811813801527023, 0.0604468397796154, 0.028961602598428726, -0.03678739443421364, -0.019380442798137665, -0.02209840714931488, 0.03920028731226921, -0.018888317048549652, -0.005084540229290724, -0.04091056436300278, -0.034725554287433624, 0.046755511313676834, -0.08005115389823914, 0.04272235557436943, 0.03135533258318901, -0.01683923415839672, -0.027781138196587563, 0.07840771973133087, -0.035257428884506226, -0.007830385118722916, 0.08670549839735031, 0.036914076656103134, -0.032798100262880325, 0.020325036719441414, -0.06208853796124458, 0.017314568161964417, -0.03200206905603409, 0.06383490562438965, 0.07165518403053284, 0.05041763558983803, -0.06396691501140594, -0.08831629902124405, -0.09819215536117554, 0.07113763689994812, -0.02043493092060089, 0.0488685742020607, 0.041572973132133484, 0.038542572408914566, 0.036640431731939316, -0.07621290534734726, 0.010468573309481144, 0.052478693425655365, -0.020086880773305893, 0.07071282714605331, -0.037182994186878204, -0.06328681111335754, -0.01058516651391983, 0.039797961711883545, 0.02846328355371952, 0.04233362525701523, -0.11232448369264603, -0.1269526481628418, -0.07002739608287811, 0.018808159977197647, 0.06420274823904037, 0.11160807311534882, -0.03576426953077316, 0.07398945093154907, -0.008989060297608376, -0.0524916872382164, -0.0006893235258758068, 0.08746752887964249, 0.042928896844387054, 0.09697806090116501, -0.03368643298745155, 0.016717134043574333, -0.06737484037876129, 0.11117848008871078, -0.0064667509868741035, 0.017092587426304817, -0.028428388759493828, 0.07917037606239319, 0.023049399256706238, 0.10372789204120636, -0.07073163986206055, -0.012214861810207367, 0.0055601466447114944, -0.060134679079055786, -0.11580318212509155, 0.05528956651687622, 0.019843164831399918, 0.05644454434514046, 0.08181146532297134, 0.018538417294621468, -0.04360484704375267, 0.003939340356737375, 0.0029382151551544666, 0.06832027435302734, 0.03265935927629471, 0.0021365126594901085, 0.08288965374231339, 0.08589732646942139, -0.0015229517593979836, 0.051206205040216446, 0.0008330842247232795, -0.0883234441280365, -0.05762103945016861, -0.0811489075422287, -0.08463174104690552, -0.06365478038787842, 0.02946806140244007, 0.008423276245594025, 0.07290886342525482, 0.06017094850540161, -0.06101946905255318, -0.10711859911680222, 0.005014287773519754, 0.0793047621846199, -0.04724366217851639, 0.0979984700679779, -0.035610198974609375, 0.00620647519826889, 0.06898141652345657, 0.0008540180861018598, 0.04411301761865616, -0.013142545707523823, 0.04710840433835983, 0.033806998282670975, 0.05543823167681694, -0.08661741763353348, 0.07892551273107529, 0.0022624770645052195, 0.03789781406521797, 0.03880668058991432, 0.10982591658830643, 0.05385102331638336, -0.07123733311891556, 0.04865161329507828, 0.10266661643981934, 0.06478849053382874, -0.03230290114879608, -0.07728026807308197, 0.03885297849774361, -0.06352828443050385, 0.03382503613829613, 0.05586228519678116, 0.10094910860061646, 0.03646277263760567, 0.1045994982123375, -0.022802162915468216, -0.04201670363545418, 0.0208229161798954, -0.06191180646419525, -0.10973493754863739, -0.0718858614563942, 0.04810023307800293, 0.08956277370452881, -0.08556058257818222, -0.08236371725797653, 0.02172115258872509, 0.06739071756601334, 0.060866955667734146, 0.006846804637461901, 0.10578997433185577, -0.0058323233388364315, 0.10075079649686813, 0.024553237482905388, -0.0740475133061409, -0.01116268988698721, -0.031305622309446335, 0.056795742362737656, 0.07214798033237457, -0.036855198442935944, -0.022399505600333214, 0.010993091389536858, 0.0768439918756485, -0.03233599662780762, 0.007047292310744524, -0.06524071097373962, -0.1100057065486908, 0.076522596180439, -0.021942907944321632, 0.091759093105793, -0.019055794924497604, -0.03362631797790527, 0.05384788289666176, -0.07103100419044495, -0.023021316155791283, 0.008463038131594658, 0.027717310935258865, -0.04404761269688606, 0.03871744126081467, 0.08407039940357208, -0.000385978608392179, 0.04429054260253906, -0.03425572067499161, -0.10339710861444473, -0.06581861525774002, -0.06646756827831268, -0.0274366345256567, -0.10805162042379379, -0.06468266993761063, -0.023900218307971954, 0.04755094274878502, -0.051602017134428024, -0.06417936086654663, -0.035311657935380936, 0.009318510070443153, 0.0161282941699028, -0.08089353144168854, -0.06328246742486954, -0.08017366379499435, -0.0346670038998127, 0.07523387670516968, 0.06295516341924667, 0.04729907214641571, -0.019745422527194023, -0.03219091147184372, -0.037725336849689484, 0.010064585134387016, 0.05700090527534485, -0.023707013577222824, 0.09146377444267273, -0.053350791335105896, 0.07422955334186554, -0.0006231426959857345, -0.030112504959106445, -0.05640675127506256, 0.05675676837563515, -0.08274909853935242, 0.050684891641139984, 0.06611742824316025, 0.024877889081835747, 0.06887662410736084, -0.061142828315496445, 0.08724240958690643, 0.039413101971149445, -0.024910690262913704, -0.033275604248046875, 0.0888696238398552, -0.05082803592085838, 0.0712859258055687, 0.05784493312239647, -0.00022792276286054403, 0.019805191084742546, 0.0023537136148661375, 0.051328402012586594, 0.024932589381933212, -0.08653494715690613, -0.01722705364227295, -0.03932027891278267, -0.026883719488978386, -0.08372033387422562, 0.059693653136491776, 0.06592682749032974, 0.059318359941244125, 0.0008204004843719304, -0.07205649465322495, 0.023437796160578728, -0.01921873167157173, -0.023775536566972733, -0.012118055485188961, -0.0015393306966871023, 0.06692102551460266, -0.06120820343494415, -0.006697484757751226, 0.003711664117872715, -0.026657799258828163, -0.02233509160578251, -0.000781460665166378, 0.003784800413995981, 0.009241999126970768, 0.003238887060433626, -0.0024135990533977747, -0.08928462862968445, 0.023920077830553055, 0.01998119428753853, -0.010642610490322113, 0.04104464873671532, -0.06827495992183685, 0.08329460024833679, -0.09626486152410507, -0.031241290271282196, -0.001447637565433979, -0.0036280641797930002, -0.05520984157919884, 0.05120232701301575, -0.025305701419711113, -0.048620354384183884, -0.032888416200876236, -0.08159108459949493, -0.05357854813337326, 0.07582582533359528, 0.05354829132556915, -0.007485376205295324, 0.02851533703505993, -0.07157263904809952, 0.03844621405005455, -0.03209123760461807, -0.03726516664028168, -0.01970929279923439, -0.05455872043967247, 0.003444543108344078, 0.030619312077760696, -0.05888070538640022, -0.021247683092951775, 0.07005485892295837, -0.028868576511740685, 0.06013932079076767, -0.020637182518839836, 0.002636682242155075, 0.07085908949375153, -0.0032019810751080513, -0.018914809450507164, -0.01769031211733818, -0.05053327605128288, 0.024551725015044212, 0.021123025566339493, -0.06278868019580841, -0.059099987149238586, 0.05997498333454132, -0.06981509923934937, -0.06824284791946411, 0.04859374463558197, 0.010217766277492046, 0.03299286589026451, 0.03155195713043213, 0.041362594813108444, 0.050472062081098557, -0.04828186705708504, -0.06397734582424164, -0.004812449216842651, 0.0016188197769224644, -0.0048774187453091145, 0.05456366762518883, 0.055832672864198685, 0.07229487597942352, -0.023759491741657257, -0.05343271419405937, 0.036837704479694366, -0.09608645737171173, -0.0699581727385521, -0.07798974961042404, 0.04725346714258194, 0.08404888212680817, 0.04870922863483429, -0.10831630229949951, 0.0036109585780650377, 0.030765971168875694, -0.06878061592578888, -0.05101441591978073, -0.0716935321688652, -0.06656817346811295, -0.021678365767002106, 0.012337353080511093, -0.08591967821121216, -0.011100353673100471, 0.08769755810499191, 0.0631955936551094, 0.045305464416742325, 0.053575299680233, 0.010826452635228634, -0.05838323011994362, 0.039797261357307434, -0.04349039867520332, 0.00015381647972390056, 0.019883206114172935, -0.0068300398997962475, 0.028529318049550056, -0.0935821384191513, -0.058357998728752136, 0.017184481024742126, -0.04823760315775871, 0.0014715520665049553, 0.03859391063451767, 0.007136036641895771, -0.07625886052846909, -0.042073704302310944, -0.09864930063486099, 0.085719034075737, 0.0012661345535889268, -0.053007565438747406, 0.0007769184885546565, -0.04079750180244446, 0.016464322805404663, -0.08312445133924484, 0.028608232736587524, 0.010336906649172306, 0.032755542546510696, 0.06629881262779236, 0.05067000165581703, -0.026752488687634468, 0.05008360743522644, 0.06814345717430115, 0.06512995809316635, 0.09111303836107254, 0.05048026144504547, -0.05933661013841629, -0.10532667487859726, 0.048545997589826584, -0.016621381044387817, -0.08417908102273941, -0.05762660503387451, 0.04923851042985916, 0.06345181912183762, -0.019275669008493423, -0.09052471071481705, -0.025563301518559456, -0.11210128664970398, -0.051421329379081726, 0.00517593789845705, -0.0025144980754703283, -0.04108019173145294, 0.054944876581430435, 0.004668904934078455, 0.0035128185991197824, 0.06101366505026817, -0.027719978243112564, -0.04851771518588066, -0.03427436575293541, -0.022285185754299164, -0.03306974098086357, -0.05538840964436531, 0.05746905505657196, 0.027052681893110275, -0.00028393446700647473, 0.0295061394572258, -0.035005949437618256, 0.07081470638513565, 0.01618904247879982, -0.03257542848587036, -0.034972213208675385, -0.010868418030440807, -0.04964728280901909, -0.06699273735284805, -0.0534207709133625, -0.008955555036664009, -0.05638225004076958, -0.03820578008890152, -0.01634492538869381, 0.0011059925891458988, -0.019855758175253868, 0.004593804944306612, 0.04485639929771423, -0.03989739716053009, -0.03212815895676613, 0.003547125728800893, -0.04329425469040871, 0.020053155720233917, 0.0130917401984334, 0.0021289787255227566, -0.043377477675676346, 0.08723417669534683, 0.07717999815940857, -0.05667852237820625, 0.06122869253158569, 0.009232653304934502, 0.032268840819597244, -0.05560389161109924, 0.06684048473834991, -0.0605287067592144, 0.01083603035658598, -0.05086623504757881, -0.05636726692318916, -0.005276942625641823, 0.029569566249847412, -0.036354321986436844, -0.05828021466732025, 0.03928746283054352, -0.05349688604474068, -0.027915803715586662, 0.003775181947275996, -0.012040093541145325, 0.041612450033426285, 0.04391563683748245, -0.05124492570757866, 0.07203728705644608, 0.0468968041241169, -0.05719960480928421, 0.02223992347717285, -0.07708238810300827, -0.004216374363750219, -0.03533120080828667, -0.08167534321546555, -0.04645426198840141, -0.017929822206497192, -0.029914967715740204, 0.012851139530539513, -0.06401532143354416, 0.024977821856737137, -0.09113997220993042, 0.014604144729673862, 0.07767783105373383, 0.011317565105855465, -0.06228809803724289, 0.011949429288506508, -0.061368122696876526, 0.04229885712265968, -0.020355012267827988, -0.020103536546230316, 0.03393120691180229, -0.016166850924491882, 0.0045712776482105255, 0.018519170582294464, -0.09684238582849503, -0.017439089715480804, 0.04421982914209366, -0.0419783741235733, 0.04574112594127655, -0.04231538251042366, 0.048375532031059265, 0.08907780051231384, -0.08604497462511063, -0.07156600803136826, -0.08593317121267319, 0.07173315435647964, -0.022528447210788727, 0.0695989578962326, -0.033737193793058395, -0.06440145522356033, 0.013218667358160019, 0.07991688698530197, 0.05312645435333252, 0.038676027208566666, -0.028214238584041595, -0.026323208585381508, 0.031488627195358276, 0.0034570691641420126, 0.06303760409355164, -0.053066134452819824, -0.06638679653406143, -0.03957311809062958, 0.03515143692493439, 0.05690636485815048, -0.06394076347351074, -0.05770230293273926, -0.020043538883328438, -0.01331411860883236, 0.017386551946401596, -0.07301495969295502, 0.08274079114198685, -0.047295503318309784, 0.07270625978708267, -0.07333506643772125, -0.006303344387561083, -0.04342424124479294, 0.029340077191591263, 0.026086483150720596, 0.019009528681635857, -0.050103407353162766, 0.017388980835676193, 0.02434254251420498, -0.1017996147274971, -0.05562196299433708, -0.11286427825689316, -0.026951275765895844, 0.06595614552497864, -0.0300335381180048, -0.07645183801651001, -0.043942641466856, -0.03309723734855652, -0.003043507691472769, -0.06820724904537201, -0.013802757486701012, -0.031292859464883804, -0.04675891250371933, -0.02799876593053341, -0.09898490458726883, -0.016185907647013664, -0.04927821829915047, 0.09932337701320648, -0.014658480882644653, -0.041424062103033066, -0.0910540372133255, -0.019502006471157074, 0.008262509480118752, 0.026367513462901115, 0.007493984419852495, 0.034815482795238495, -0.037334881722927094, -0.07922058552503586, -0.046358417719602585, -0.07938308268785477, -0.03474920615553856, 0.04501422122120857, -0.03495746850967407, -0.11478451639413834, 0.046965114772319794, 0.0022834953851997852, -0.054815758019685745, 0.05090193822979927, -0.006279582157731056, 0.003203814150765538, 0.04009076580405235, -0.05138860642910004, -0.028678180649876595, 0.014913961291313171, 0.07315196096897125, -0.023112701252102852, 0.09379272907972336, 0.030213747173547745, -0.0330740250647068, -0.0632786676287651, -0.055795591324567795, 0.03927890583872795, 0.0379784032702446, 0.05254967138171196, -0.07071560621261597, -0.0014893489424139261, -0.0211119893938303, -0.07605806738138199, 0.055205222219228745, -0.018071182072162628, 0.09239310026168823, -0.010858503170311451, -0.001745869405567646, 0.018461421132087708, -0.012564794160425663, 0.06727319955825806, -0.07963807880878448, -0.024801094084978104, -0.09281496703624725, 0.038278017193078995, -0.09095233678817749, -0.01077879685908556, -0.016397003084421158, -0.08079664409160614, -0.03555328771471977, -0.06858416646718979, 0.05991312116384506, 0.03870520740747452, 0.007264965679496527, -0.033626437187194824, -0.04571588709950447, 0.024484358727931976, 0.09457646310329437, 0.028830045834183693, -0.0416896753013134, 0.015834972262382507, -0.06759660691022873, 0.06151633337140083, -0.015888387337327003, 0.03863503411412239, -0.06246279552578926, -0.053544800728559494, -0.0823635682463646, -0.020777851343154907, 0.05540824308991432, 0.03879469633102417, -0.0046791862696409225, -0.05261576175689697, -0.04139929264783859, -0.05108587443828583, 0.08769112080335617, -0.08055436611175537, 0.03883909434080124, 0.08706364035606384, -0.008665012195706367, 0.034862298518419266, 0.03271728754043579, 0.07095180451869965, -0.08713915199041367, 0.0249311625957489, -0.007360920775681734, -0.015111207962036133, 0.01890791393816471, -0.05322262644767761, -0.05625775456428528, -0.10093599557876587, 0.0433298759162426, 0.0029722701292485, 0.028087619692087173, 0.02545168064534664, -0.05181773751974106, -0.05020706728100777, -0.05575468763709068, 0.03232750669121742, -0.04160129651427269, 0.0929877758026123, -0.0361969955265522, -0.020473161712288857, 0.07232622802257538, -0.0531197227537632, -0.08624637126922607, 0.015961850062012672, -0.004982902202755213, 0.05632675439119339, 0.03963197395205498, -0.018120035529136658, 0.0161297470331192, -0.011744262650609016, -0.06822511553764343, 0.02944585494697094, -0.0015503723407164216, 0.09360963106155396, 0.11360606551170349, 0.00027244159718975425, -0.09166689962148666, 0.05402332916855812, 0.037837352603673935, -0.0485377199947834, -0.08968866616487503, -0.08941613882780075, -0.0490969680249691, -0.058962542563676834, 0.06301427632570267, 0.029113972559571266, -0.004219912923872471, -0.03399953246116638, 0.09239508956670761, 0.08269721269607544, 0.07154349237680435, 0.02051496133208275, -0.006752246059477329, -0.04117899388074875, 0.07491005212068558, 0.0003489681985229254, 0.039458490908145905, 0.03539978712797165, 0.05211258679628372, -0.0019021944608539343, -0.046623680740594864, 0.07212714105844498, -0.07786960154771805, -0.03895329684019089, -0.033239006996154785, -0.0345555879175663, -0.07550545781850815, 0.06740852445363998, -0.07761529833078384, 0.05490071699023247, 0.029055902734398842, 0.0007605772116221488, 0.01891116239130497, 0.012912388890981674, 0.03738364204764366, -0.02584231086075306, 0.04296138137578964, -0.05323203653097153, 0.06881049275398254, 0.06305564939975739, -0.02472965233027935, 0.009050732478499413, 0.021567417308688164, -0.0419263131916523, -0.02759043127298355, 0.04571796953678131, 0.06619208306074142, -0.0613955482840538, 0.03308386355638504, -0.055583320558071136, -0.054758600890636444, 0.006755135953426361, 0.007810491137206554, -0.04173041135072708, -0.020272858440876007, 0.057771630585193634, -0.029580049216747284, 0.05907342955470085, -0.005805688444525003, 0.04581999406218529, 0.08275987952947617, 0.037025611847639084, 0.0032827984541654587, 0.06902164965867996, 0.01388388779014349, -0.07138074934482574, -0.005077915266156197, 0.07970666140317917, 0.05843646079301834, 0.027989409863948822, -0.03130157291889191, 0.06361168622970581, 0.05128548666834831, -0.06912118941545486, -0.03550483658909798, -0.07688210159540176, -0.0040172976441681385, 0.06615428626537323, 0.057273298501968384, 0.0675799697637558, 0.03780760243535042, -0.059685833752155304, -0.05418533831834793, -0.03676192834973335, -0.052093397825956345, -0.10185059159994125, -0.0411902517080307, 0.07219536602497101, 0.02396344766020775, 0.05262508615851402, 0.03620023652911186, -0.09633149951696396, -0.05945910885930061, 0.06613734364509583, 0.036237213760614395, -0.00841451808810234, -0.0011568272020667791, -0.007803805638104677, 0.0755104273557663, 0.001921548624522984, -0.00929025374352932, -0.06538813561201096, -0.04270675778388977, -0.024932347238063812, -0.09110505878925323, -0.03557836636900902, -0.09487178921699524, -0.014594441279768944, 0.06483929604291916, -0.09879015386104584, 0.05712415650486946, -0.05667683109641075, 0.04496343433856964, 0.09118866175413132, -0.016685493290424347, 0.04357745498418808, 0.004550079349428415, 0.017159096896648407, -0.008807690814137459, -0.00666736951097846, 0.049394652247428894, -0.06882515549659729, -0.03424284979701042, -0.07344553619623184, 0.0761183574795723, -0.0731763243675232, 0.021967662498354912, 0.004080340266227722, -0.05401794984936714, 0.04596533626317978, -0.04994227737188339, -0.010006972588598728, 0.05289202183485031, -0.037478744983673096, 0.020878123119473457, 0.04261196032166481, -0.016886960715055466, -0.024654515087604523, -0.07382696121931076, -0.09260398894548416, -0.03969525173306465, -0.07512804865837097, 0.021032387390732765, 0.06517598032951355, 0.014176618307828903, -0.030288169160485268, 0.07523920387029648, 0.04188988730311394, 0.0515231117606163, 0.0172007754445076, 0.012606444768607616, -0.09017671644687653, -0.03631149232387543, -0.01119180116802454, -0.07787224650382996, -0.04677337035536766, -0.007861053571105003, -0.038094308227300644, -0.05506446585059166, 0.030905134975910187, 0.03370317444205284, -0.04501469060778618, 0.057704418897628784, -0.04769324138760567, 0.021113654598593712, -0.0260329432785511, -0.023183370009064674, 0.0821959376335144, -0.07625720649957657, -0.04138907045125961, -0.030283350497484207, 0.08838696777820587, -0.012474136427044868, -0.07612897455692291, -0.03129302337765694, 0.058330945670604706, -0.002764662727713585, 0.04384384676814079, -0.016362465918064117, 0.0658097192645073, -0.0460110679268837, 0.03860948607325554, 0.002418092917650938, 0.07548978179693222, -0.05764755606651306, 0.07845887541770935, -0.03194813057780266, -0.073701873421669, 0.04208938777446747, 0.03932100906968117, 0.05263269692659378, 0.0651104748249054, -0.0031418772414326668, -0.017274390906095505, 0.005797530524432659, 0.02446790412068367, 0.053554680198431015, 0.005438240710645914, -0.09191038459539413, -0.09238357841968536, 0.009845001623034477, -0.0915088802576065, -0.013240714557468891, -0.03404964879155159, 0.04934144765138626, -0.05005885288119316, -0.043598029762506485, 0.07923318445682526, 0.03606167435646057, 0.012278351932764053, 0.04411211237311363, -0.02220117673277855, 0.07548093795776367, -0.12270637601613998, -0.014092938974499702, -0.004685047548264265, 0.07478272914886475, 0.0748835876584053, 0.06252124160528183, -0.0546107292175293, -0.039800796657800674, -0.016344470903277397, 0.05374443158507347, 0.0792890191078186, 0.06022151932120323, -0.06299664825201035, 0.07383603602647781, 0.009823310188949108, 0.019228830933570862, -0.07007655501365662, -0.04716268181800842, -0.04482227563858032, 0.05647134780883789, 0.09277848154306412, -0.043809302151203156, -0.020631995052099228, 0.02239788882434368, -0.0840827226638794, 0.06437233835458755, -0.005774473771452904, -0.02267535589635372, -0.038168247789144516, -0.020719094201922417, 0.06179186701774597, 0.010838315822184086, -0.023035423830151558, -0.07724262773990631, 0.041111499071121216, 0.012916143983602524, 0.022730860859155655, 0.033332474529743195, 0.05465315282344818, 0.04046761989593506, -0.025932064279913902, 0.06447767466306686, -0.09685686975717545, 0.029330359771847725, 0.005501078441739082, 0.11073596030473709, -0.03558109700679779, 0.09965305775403976, -0.05068856105208397, 0.09603878855705261, 0.058836087584495544, -0.03897919878363609, -0.06982319802045822, 0.0298136156052351, -0.009492084383964539, -0.01705448515713215, 0.06265954673290253, 0.03349878266453743, -0.03972504287958145, -0.06356009095907211, 0.032309483736753464, 0.0378081239759922, 0.030186165124177933, -0.0644177719950676, 0.08070439845323563, 0.018701031804084778, 0.03672955930233002, 0.0008472377667203546, 0.04227176308631897, 0.09900589287281036, -0.021852467209100723, 0.013238651677966118, -0.09069930762052536, 0.0044477847404778, 0.0009087324142456055, 0.05194510519504547, 0.01253496389836073, 0.014073953032493591, 0.02100502885878086, -0.09078841656446457, -0.017223156988620758, -0.0007442189962603152, -0.053668342530727386, 0.07213550060987473, 0.00022212089970707893, -0.022491157054901123, -0.01754194311797619, -0.044717635959386826, 0.0019918682519346476, 0.02268415130674839, 0.05889454856514931, 0.007678445428609848, -0.007060876581817865, -0.08445873856544495, -0.04127286747097969, -0.08037347346544266, -0.08236049860715866, 0.033127594739198685, -0.083628810942173, -0.006403863895684481, -0.046893976628780365, -0.004343792330473661, 0.049244508147239685, -0.08145112544298172, 0.03052663989365101, 0.08930914849042892, 0.08662538230419159, 0.03790901601314545, -0.0794215202331543, -0.0746624693274498, -0.0013856866862624884, -0.08381187915802002, -0.1060098335146904, 0.002845680806785822, -0.0219102930277586, -0.054959408938884735, -0.040507834404706955, -0.007225483655929565, -0.07130705565214157, 0.042564768344163895, -0.02932499162852764, -0.024588029831647873, 0.10866396874189377, 0.0004717154079116881, 0.013245859183371067, -0.057977162301540375, -0.04531772807240486, 0.06867764890193939, 0.041499678045511246, 0.021008208394050598, -0.08063029497861862, 0.0020570328924804926, 0.05023106187582016, 0.07439179718494415, 0.01944008469581604, 0.006134794559329748, -0.01893317885696888, 0.02637375518679619, 0.04636496305465698, 0.002025562571361661, 0.039026688784360886, -0.04832855984568596, 0.04199701547622681, 0.02164210006594658, -0.03720364347100258, 0.08861041069030762, -0.00035384087823331356, 0.035297270864248276, -0.03715193271636963, -0.07467017322778702, 0.06890261173248291, -0.07914083451032639, 0.10313389450311661, 0.024598287418484688, 0.04885534197092056, -0.07634822279214859, -0.028516825288534164, -0.07827994227409363, -0.008051722310483456, 0.0436549074947834, -0.02400999888777733, 0.007474167738109827, -0.021370187401771545, 0.12070747464895248, 0.02064545825123787, 0.06773695349693298, -0.055316291749477386, 0.06357668340206146, 0.052059415727853775, 0.031699616461992264, -0.005106661468744278, -0.0286006610840559, 0.08373002707958221, 0.0485844761133194, -0.03348338231444359, -0.010630952194333076, 0.020215509459376335, 0.02442929707467556, 0.03521113097667694, -0.0027119950391352177, -0.002948036417365074, -0.05050446838140488, -0.031943127512931824, 0.04570745304226875, -0.01176109816879034, 0.02414952591061592, 0.015660446137189865, -0.08195473253726959, 0.019407490268349648, 0.08382707089185715, -0.05122896283864975, 0.01514439843595028, -0.07079020887613297, -0.06085365638136864, -0.00207504746504128, -0.0571276880800724, -0.07785508781671524, -0.04112416133284569, -0.013225078582763672, -0.04113088548183441, -0.08450234681367874, 0.07793210446834564, -0.08906111121177673, -0.04180154949426651, -0.062762051820755, 0.02362944558262825, -0.05494644120335579, 0.0217761117964983, 0.019602032378315926, -0.027033325284719467, -0.05809306725859642, 0.07006611675024033, 0.03439318761229515, -0.020493533462285995, 0.052654169499874115, -0.015466875396668911, -0.06104751303792, 0.003756995778530836, 0.05211638659238815, 0.00809354055672884, 0.03721228614449501, 0.033994246274232864, 0.08775343000888824, -0.01669912412762642, 0.03161250427365303, 0.07709503918886185, -0.06040068343281746, -0.015381261706352234, 0.024040935561060905, 0.01048140600323677, -0.042436402291059494, 0.0611749105155468, -0.06038953363895416, 0.014599435031414032, -0.037257906049489975, -0.09231160581111908, -0.06185105815529823, 0.05041968449950218, 0.013782047666609287, 0.015693677589297295, 0.031058911234140396, -0.0673963725566864, -0.0659538060426712, -0.059383638203144073, -0.024438777938485146, 0.03565441817045212, 0.05645272880792618, -0.03931761533021927, 0.0038666194304823875, -0.05226748064160347, 0.06822460144758224, 0.07172410190105438, -0.032737214118242264, -0.017877455800771713, 0.06677832454442978, 0.044517744332551956, -0.07460319250822067, 0.011368569917976856, 0.042128920555114746, -0.06428469717502594, 0.05117614567279816, -0.028714105486869812, -0.08730331063270569, 0.09367930144071579, 0.06078687310218811, 0.06760340929031372, -0.043813396245241165, 0.035333286970853806, -0.08526374399662018, -0.028308138251304626, 0.006051323376595974, 0.027165507897734642, -0.022555604577064514, 0.005750140640884638, -0.08250419050455093, 0.02286209724843502, 0.04227455332875252, 0.057672496885061264, -0.025152578949928284, 0.030550111085176468, -0.046827737241983414, -0.009368657134473324, 0.013930682092905045, -0.08811210095882416, -0.07754956930875778, -0.04849648475646973, 0.024816932156682014, -0.008591989986598492, 0.0014687407528981566, 0.05741791054606438, 0.08571822941303253, -0.021676644682884216, -0.06550277769565582, -0.0130770243704319, -0.004725717008113861, -0.07032293826341629, 0.0618799664080143, -0.012806526385247707, 0.023607801645994186, -0.06045224145054817, -0.037520814687013626, -0.07263274490833282, 0.022894708439707756, 0.008896060287952423, 0.05266956612467766, -0.0422515943646431, 0.04735194519162178, -0.0679454430937767, 0.08737499266862869, -0.09162934869527817, 0.04699243605136871, -0.013657815754413605, 0.02265237085521221, -0.021163497120141983, 0.06885036081075668, -0.051403116434812546, -0.05589360371232033, 0.04929899424314499, -0.08898293972015381, 0.007745024748146534, -0.0781673938035965, -0.0376335009932518, 0.09127326309680939, -0.0007619014359079301, -0.02431352436542511, 0.03196238726377487, -0.027278045192360878, -0.05696988105773926, 0.09187737107276917, 0.021829375997185707, 0.04315747320652008, -0.034511879086494446, 0.011492297053337097, -0.055142924189567566, 0.07604268193244934, -0.037983089685440063, 0.0941208079457283, -0.019891588017344475, 0.03666830435395241, -0.03941795602440834, -0.05010661110281944, -0.037255533039569855, -0.05282377451658249, -0.05612992122769356, 0.09947958588600159, 0.06973985582590103, -0.050982117652893066, -0.022140268236398697, -0.034063175320625305, 0.047719210386276245, 0.05866086110472679, 0.040598396211862564, 0.008296473883092403, -0.04660169035196304, -0.0016882313648238778, 0.021828681230545044, -0.0439600944519043, -0.06148598715662956, -0.0065475646406412125, -0.019108528271317482, 0.0631781816482544, -0.03176811337471008, -0.012838308699429035, 0.04577381908893585, 0.031632084399461746, -0.018993796780705452, -0.051572225987911224, 0.08017858117818832, 0.06297668069601059, -0.010827960446476936, -0.07613563537597656, 0.0066932328045368195, 0.06713675707578659, 0.037606120109558105, -0.04391457512974739, -0.07083771377801895, -0.08115799725055695, -0.027704959735274315, 0.05921817943453789, -0.11431290209293365, -0.02619750052690506, -0.04751969873905182, -0.07485400885343552, -0.06682050228118896, -0.010042058303952217, 0.03778582438826561, -0.03019293211400509, -0.029800627380609512, -0.012174161151051521, -0.012862044386565685, 0.015231522731482983, -0.05128183215856552, 0.08046147227287292, 0.06098226085305214, -0.006259238347411156, -0.028010893613100052, 0.07566987723112106, 0.09969060868024826, 0.04250907897949219, 0.04398815706372261, -0.03513482213020325, -0.06480783969163895, -0.04471337050199509, -0.06936602294445038, 0.026032879948616028, -0.0004901009961031377, 0.048084892332553864, 0.06171983480453491, -0.006114158779382706, -0.06152007728815079, 0.03816010430455208, -0.002906407695263624, -0.05172838270664215, 0.09132373332977295, 0.03246054798364639, 0.06684613972902298, -0.03934340178966522, -0.06247831881046295, 0.0521533228456974, 0.03945615887641907, 0.058198414742946625, -0.09944727271795273, -0.0803430825471878, 0.05878366529941559, 0.08509650081396103, -0.03311242163181305, -0.012436127290129662, -0.09080195426940918, -0.10192227363586426, -0.030936138704419136, -0.08647630363702774, -0.05831501632928848, -0.03978723660111427, 0.018277404829859734, 0.04727168753743172, 0.014814740978181362, 0.045035555958747864, 0.09799547493457794, -0.034677308052778244, 0.01894865557551384, -0.041241906583309174, 0.02999846264719963, -0.06710216403007507, -0.0766265019774437, 0.02671351283788681, 0.047323256731033325, 0.08278165757656097, -0.04168345779180527, 0.048419564962387085, 0.0008874546620063484, -0.018631499260663986, -0.00265008257701993, -0.03357619047164917, 0.07441867887973785, -0.02711229771375656, -0.08804644644260406, -0.0814943015575409, 0.04548341780900955, -0.00457251351326704, -0.05458875000476837, -0.0865604504942894, -0.0552937313914299, -0.1084781140089035, -0.0012673843884840608, -0.04366176575422287, 0.02376445196568966, -0.08432788401842117, 0.0555146224796772, 0.0936780646443367, -0.052008580416440964, 0.10651317983865738, 0.0001567421859363094, -0.02016100473701954, -0.030154500156641006, -0.06596415489912033, -0.04133245348930359, -0.04185923561453819, -0.07522609829902649, -0.014013978652656078, 0.004483354743570089, 0.02976752072572708, 0.11158046126365662, -0.04659892991185188, 0.06737268716096878, 0.0005347684491425753, -0.05594192072749138, 0.006886172574013472, -0.003860448719933629, 0.01657896116375923, 0.0922049805521965, 0.020856009796261787, 0.04338579252362251, 0.06011276692152023, 0.02837429568171501, -0.07093487679958344, -0.037534698843955994, -0.07884000241756439, 0.06001901254057884, 0.11024469882249832, 0.05805625393986702, -0.08798027038574219, -0.06567807495594025, -0.10824015736579895, -0.024683523923158646, 0.02344435080885887, 0.11661219596862793, 0.06473787128925323, -0.015207376331090927, -0.0002640982565935701, -0.03586600720882416, -0.054559607058763504, -0.03138154000043869, 0.011863164603710175, 0.11003586649894714, -0.05236256122589111, -0.08654003590345383, -0.05407265946269035, -0.014512433670461178, 0.058433566242456436, -0.024068616330623627, 0.059613797813653946, -0.04854169860482216, -0.03952159360051155, 0.03154616430401802, -0.0668104812502861, -0.07756318151950836, -0.032777268439531326, 0.08443978428840637, 0.0201196800917387, 0.06408444046974182, -0.03512771800160408, -0.0011462988331913948, -0.03347072750329971, -0.037119533866643906, 0.006401205901056528, 0.01220411155372858, 0.02493220753967762, -0.0920620858669281, -0.016183331608772278, 0.10376758128404617, -0.03359321877360344, -0.09158425778150558, -0.05472763255238533, -0.05179986730217934, 0.10500051081180573, -0.02340126410126686, 0.05974468216300011, 0.08298324048519135, -0.0159507617354393, 0.017148274928331375, -0.08378230035305023, 0.07751990109682083, 0.039000220596790314, 0.04722793772816658, -0.10719674080610275, -0.006834699772298336, 0.03479991853237152, 0.0664011761546135, -0.03602497652173042, 0.07704321295022964, -0.0023456241469830275, 0.006507967598736286, 0.006366933695971966, -0.026326213032007217, -0.013543875887989998, -0.04878722131252289, 0.05594298988580704, 0.05767988786101341, 0.023435574024915695, 0.00816799234598875, -0.03626358509063721, 0.07224757969379425, 0.08634451031684875, -0.01312375906854868, -0.05096322298049927, 0.06393125653266907, -0.06436032056808472, 0.06077948212623596, 0.052668068557977676, -0.0029571978375315666, 0.011076190508902073, 0.038191135972738266, 0.02666384167969227, 0.04413781687617302, -0.05386282876133919, 0.061033058911561966, 0.022989897057414055, -0.09455325454473495, -0.009224249981343746, -0.05934417247772217, -0.001859954558312893, 0.037284526973962784, 0.060626424849033356, -0.0685223862528801, 0.036252330988645554, -0.07427771389484406, 0.011604552157223225, -0.011893310584127903, 0.022354332730174065, 0.014905955642461777, -0.011640841141343117, 0.08275844901800156, 0.011461182497441769, -0.04519108682870865, -0.03533082827925682, 0.07468153536319733, 0.07118005305528641, -0.11193080991506577, -0.039733242243528366, 0.021868446841835976, 0.020985038951039314, 0.0787489116191864, 0.07366558164358139, 0.05049651861190796, 0.026712587103247643, 0.03465476632118225, -0.034243084490299225, -0.03579903766512871, -0.05655217543244362, 0.03448151797056198, 0.03811751306056976, 0.028842104598879814, 0.05799435079097748, -0.08855042606592178, 0.06271157413721085, 0.019869834184646606, -0.032922275364398956, 0.06316430121660233, 0.08344343304634094, -0.06752285361289978, -0.05411370098590851, -0.06371348351240158, -0.004083994310349226, -0.045592907816171646, -0.09935508668422699, -0.09035514295101166, -0.07009910047054291, 0.07568777352571487, -0.06967518478631973, 0.051474783569574356, 0.08419644832611084, -0.04201383888721466, 0.043179702013731, -0.0798555463552475, -0.017067883163690567, -0.02179277315735817, -0.08240094035863876, -0.002619246719405055, 0.029095251113176346, -0.0006009297794662416, 0.014874747954308987, 0.025172501802444458, 0.01739233359694481, 0.020281169563531876, -0.01989833265542984, 0.0770949274301529, 0.040234193205833435, 0.05654587596654892, -0.04463496431708336, -0.04136398807168007, 0.06119227036833763, -0.08968164026737213, 0.05214015394449234, -0.0479699969291687, -0.010299489833414555, 0.016495676711201668, 0.10076535493135452, -0.09559118747711182, -0.08042596280574799, -0.0898650735616684, 0.046549975872039795, -0.02384340949356556, 0.048928339034318924, -0.07400596886873245, 0.00424617063254118, -0.05918310210108757, 0.06863364577293396, -0.10631947964429855, 0.08129318058490753, 0.018360398709774017, 0.08418412506580353, -0.0953458622097969, 0.020011043176054955, -5.899257666897029e-05, 0.05165086314082146, -0.01443480048328638, -0.021933604031801224, 0.06973922997713089, -0.01841251365840435, 0.07918284088373184, -0.06835722178220749, 0.07527031004428864, 0.04350699484348297, -0.014628734439611435, -0.0329718254506588, 0.051716871559619904, -0.0790187418460846, 0.03591610863804817, -0.10363325476646423, 0.05946492776274681, -0.09231115877628326, 0.08644367009401321, 0.025047242641448975, 0.06088925525546074, -0.00522031681612134, 0.09520290791988373, -0.032015424221754074, 0.015801412984728813, -0.004505431279540062, -0.03824012726545334, 0.07783624529838562, 0.05738126114010811, 0.01722579263150692, 0.03907056897878647, -0.05979398265480995, -0.07367242872714996, 0.061353810131549835, -0.09240390360355377, -0.058706898242235184, -0.039533670991659164, -0.09549763798713684, -0.012487372383475304, 0.022333920001983643, -0.015015745535492897, -0.015294085256755352, -0.02912670001387596, 0.049463335424661636, -0.07396223396062851, 0.05789507180452347, -0.08056824654340744, 0.06793122738599777, 0.08495136350393295, 0.05557787045836449, -0.003540969453752041, 0.06409625709056854, 0.027699027210474014, 0.018216043710708618, -0.05366744101047516, -0.03798062354326248, -0.009747161529958248, -0.06831640750169754, -0.034717969596385956, -0.059582408517599106, -0.07893042266368866, -0.0641126036643982, 0.023328658193349838, 0.038498032838106155, -0.08213125914335251, -0.052977703511714935, -0.06169693171977997, -0.08511578291654587, -0.09108037501573563, 0.05388893559575081, 0.016182642430067062, 0.03730962797999382, -0.020192613825201988, 0.04030359163880348, 0.015599831007421017, -0.039022233337163925, -0.06986871361732483, 0.0388643816113472, -0.01503002643585205, 0.04266009479761124, 0.08486703038215637, 0.003617733484134078, 0.06567897647619247, -0.057702694088220596, -0.021205535158514977, -0.03177376464009285, 0.08249194920063019, 0.06470208615064621, -0.08108634501695633, 0.005967720411717892, -0.011021736077964306, 0.09713340550661087, 0.02992180548608303, -0.06707078963518143, -0.01092968787997961, 0.02011842094361782, 0.0265895314514637, 0.02506493218243122, 0.009381204843521118, 0.0008865190320648253, 0.07915380597114563, 0.0893498882651329, 0.05187977850437164, 0.01739649847149849, -0.07067511230707169, 0.04809077829122543, -0.07999330759048462, -0.026479993015527725, 0.010810842737555504, -0.04695359990000725, -0.012113590724766254, -0.05008590221405029, -0.01487688533961773, -0.07905639708042145, -0.05979691073298454, -0.07690252363681793, -0.06617680191993713, 0.07346215844154358, 0.00320415198802948, -0.025636522099375725, -0.07802703976631165, -0.017194494605064392, -0.030678648501634598, 0.01564924232661724, -0.02876945212483406, -0.03280340135097504, -0.03760998323559761, -0.06318297982215881, 0.06283505260944366, 0.04089019447565079, -0.03100350685417652, 0.014423410408198833, -0.027441399171948433, -0.037029705941677094, 0.06343892961740494, -0.0360204353928566, -0.085393026471138, 0.029040399938821793, 0.10146272927522659, 0.03541700541973114, 0.006349643692374229, -0.009537936188280582, -0.05971195176243782, -0.11103503406047821, -0.0658019408583641, -0.05603822320699692, -0.00147420319262892, -0.05745330825448036, 0.0990549623966217, -0.0028240024112164974, -0.030506247654557228, -0.06498771160840988, -0.020496033132076263, -0.0353735126554966, 0.007183120120316744, 0.03311007097363472, -0.0642142966389656, -0.017073510214686394, 0.005656572990119457, 0.03860682249069214, 0.0021043666638433933, -0.006020678207278252, 0.008282613940536976, -0.009398946538567543, 0.04753086715936661, -0.05303977057337761, 0.052635472267866135, 0.07012017071247101, 0.1140126883983612, 0.0050360700115561485, -0.08051206171512604, -0.030286600813269615, -0.0567343570291996, -0.09027469158172607, -0.06475138664245605, 0.02640007622539997, -0.09062188863754272, 0.017622871324419975, 0.04215824604034424, -0.07314438372850418, 0.05656389519572258, 0.0008441077079623938, -0.05664834380149841, 0.043243326246738434, 0.019737405702471733, 0.05875605717301369, -0.04601030424237251, 0.0635778158903122, 0.011657824739813805, -0.11601638793945312, 0.10786145180463791, 0.08312159031629562, -0.08754720538854599, -0.014315307140350342, 0.06436392664909363, -0.028669405728578568, -0.05370534583926201, -0.029944565147161484, -0.0027537550777196884, 0.030166782438755035, -0.08724841475486755, 0.0760224461555481, 0.004783388692885637, 0.05323134735226631, 0.12131596356630325, -0.0124552296474576, 0.05855317786335945, -0.06503079831600189, -0.06492511928081512, -0.0016437489539384842, 0.07374398410320282, 0.09254977852106094, -0.031175309792160988, -0.050449442118406296, 0.07941464334726334, 0.07198656350374222, -0.041366323828697205, -0.025821683928370476, 0.030983377248048782, 0.047822028398513794, 0.012057589367032051, -0.05670574679970741, 0.03394788131117821, 0.035011906176805496, -0.01419502217322588, 0.032942984253168106, 0.026630235835909843, 0.08697426319122314, 0.055475760251283646, -0.003114008344709873, 0.09927617013454437, -0.018497560173273087, -0.05839930474758148, -0.03672941401600838, -0.05795813351869583, 0.09497705847024918, -0.11520323157310486, -0.0480620302259922, 0.010525118559598923, 0.08123046159744263, -0.01835179701447487, -0.0739053413271904, -0.052378877997398376, -0.023434804752469063, -0.07944987714290619, -0.05017297342419624, 0.004961966071277857, 0.09471473097801208, 0.02517736703157425, 0.05629086121916771, -0.035439494997262955, -0.041555676609277725, 0.07430306822061539, -0.02438623271882534, 0.04452219605445862, -1.3881268387194723e-05, -0.060908518731594086, 0.04544072598218918, 0.009181135334074497, 0.0666578933596611, -0.04321454092860222, -0.053099196404218674, 0.03704162687063217, -0.09677261114120483, 0.0667366161942482, 0.0027029085904359818, 0.061358287930488586, 0.07500477135181427, 0.06843278557062149, 0.047833435237407684, 0.08599158376455307, -0.006038433872163296, 0.009331092238426208, -0.05202066898345947, -0.017078038305044174, 0.014783400110900402, -0.015019961632788181, -0.10531479120254517, -0.03118644282221794, 0.06256537139415741, -0.020894385874271393, 0.057904504239559174, 0.0424780510365963, 0.09354905784130096, 0.023822057992219925, -0.06996144354343414, 0.08165144175291061, 0.040962085127830505, 0.04854157939553261, -0.0003022266610059887, 0.040831468999385834, 0.08915506303310394, 0.01688116416335106, 0.026024075224995613, -0.06218448653817177, 0.007980737835168839, 0.06402808427810669, -0.006411876995116472, 0.06214914470911026, -0.039314158260822296, -0.07468772679567337, -0.03481677919626236, 0.1002291813492775, -0.06799566745758057, 0.033459145575761795, -0.038327448070049286, -0.011055459268391132, -0.06353917717933655, -0.0644788071513176, 0.08304671943187714, 0.03456420078873634, -0.007007744163274765, 0.10383790731430054, 0.07203248888254166, -0.010126428678631783, -0.09404431283473969, -0.035071153193712234, 0.08505706489086151, -0.038929227739572525, -0.06486283987760544, 0.0055268146097660065, 0.0863272175192833, -0.0029228385537862778, -0.04609491676092148, -0.029399242252111435, -0.030526544898748398, -0.08686219155788422, 0.04085231199860573, -0.045791421085596085, 0.015723830088973045, 0.06855984777212143, 0.045213282108306885, -0.019394977018237114, 0.011139709502458572, 0.011796806938946247, -0.110321544110775, 0.02298479527235031, 0.00591852655634284, -0.058499906212091446, -0.04300235956907272, 0.01965237408876419, -0.066777303814888, -0.07985579967498779, -0.049469515681266785, -0.02441267855465412, -0.03572717681527138, -0.01980188861489296, -0.004178606439381838, -0.04661474749445915, -0.014070454053580761, -0.021075785160064697, 0.06544291973114014, 0.017563771456480026, -0.05735186114907265, -0.024378716945648193, -0.04881957545876503, -0.03140581399202347, -0.06898095458745956, 0.04468372464179993, -0.05104571580886841, 0.07593293488025665, -0.057312097400426865, 0.01872325874865055, -0.07674095779657364, -0.03547016531229019, -0.0001941354712471366, 0.026232734322547913, -0.07417692244052887, 0.020408445969223976, -0.08598685264587402, -0.0133350919932127, -0.024779826402664185, -0.023206746205687523, -0.04517051950097084, -0.07055488973855972, -0.03816673532128334, 0.05657408758997917, -0.01769673079252243, 0.020189566537737846, 0.02988343872129917, 0.04857039451599121, -0.08450965583324432, -0.015803392976522446, -0.08853752911090851, -0.0482012964785099, -0.01747988536953926, -0.03564808890223503, -0.046711672097444534, -0.022228030487895012, -0.07939644902944565, -0.02046954818069935, 0.04943997412919998, -0.06983285397291183, -0.06497025489807129, -0.0551924966275692, -0.043713394552469254, -0.05354217067360878, -0.0993872880935669, 0.02159404195845127, -0.04187856614589691, -0.008346712216734886, 0.024544574320316315, -0.02036520279943943, 0.043774619698524475, 0.0576905757188797, -0.02000799961388111, 0.08377353847026825, -0.009510817006230354, 0.01287413015961647, -0.10016916692256927, 0.03351735323667526, -0.057118333876132965, -0.05783092975616455, 0.05476956441998482, 0.006159652024507523, -0.017118865624070168, -0.08348160982131958, 0.06393231451511383, 0.06020296365022659, -0.04692395031452179, -0.067459337413311, 0.01363253965973854, 0.07978146523237228, 0.09552931785583496, -0.010699250735342503, 0.043576017022132874, -0.028793124482035637, 0.10302884876728058, -0.039892107248306274, 0.057857632637023926, -0.07921434193849564, -0.07335080951452255, 0.06482825428247452, -0.06799201667308807, 0.07877367734909058, -0.003663658630102873, -0.07737714052200317, 0.035182300955057144, 0.010904895141720772, -0.07088674604892731, -0.02559378556907177, -0.0017130321357399225, 0.07203252613544464, -0.06736525148153305, -0.1006823256611824, -0.01522506307810545, -0.036780282855033875, -0.01807357743382454, -0.008966962806880474, 0.01730082370340824, 0.006274125538766384, -0.09015262871980667, 0.0173577219247818, -0.02896762639284134, -0.01092590019106865, 0.023668009787797928, -0.08230295032262802, 0.03235499933362007, 0.08417367935180664, -0.04975587874650955, 0.06263365596532822, 0.0804302990436554, 0.06289149820804596, 0.05166225507855415, -0.014731823466718197, -0.03666584938764572, -0.08453670889139175, 0.08064534515142441, -0.08891503512859344, -0.01841057650744915, 0.09575280547142029, -0.019128071144223213, -0.1073218435049057, 0.05434728413820267, 0.049404677003622055, 0.021755916997790337, 0.008659620769321918, -0.0533992275595665, -0.0026767649687826633, 0.06373301893472672, 0.05856781825423241, 0.05028846114873886, 0.029553120955824852, 0.01717992313206196, -0.09651760011911392, -0.05818541720509529, 0.055616311728954315, -0.050174545496702194, 0.010922012850642204, -0.04552874341607094, -0.05164765939116478, 0.021415207535028458, -0.07334844022989273, -0.0335986353456974, 0.030231516808271408, -0.05882777273654938, -0.04434714838862419, 0.006969854701310396, -0.06833311170339584, 0.004843252710998058, 0.02707073837518692, -0.05905158072710037, 0.04380718618631363, -0.03170131891965866, 0.012557242065668106, -0.09657371789216995, -0.06151137501001358, -0.12095513939857483, 0.06635792553424835, -0.06650131195783615, 0.029283149167895317, -0.06494304537773132, 0.036786723881959915, 0.03764508664608002, 0.018897168338298798, 0.024461522698402405, -0.07123944163322449, 0.08012586086988449, -0.1099107638001442, 0.055930133908987045, -0.014483115635812283, 0.01699083484709263, -0.060578081756830215, -0.09962578117847443, -0.040136635303497314, 0.0028402325697243214, 0.02822958305478096, 0.03135979175567627, -0.022684212774038315, 0.06285347044467926, -0.03363141790032387, -0.034289538860321045, -0.03982877731323242, -0.057457271963357925, 0.017435500398278236, 0.06911178678274155, -0.09560921043157578, 0.06040947139263153, -0.018437491729855537, -0.03868923336267471, 0.05984148010611534, 0.0699368491768837, 0.050777629017829895, -0.010656136088073254, -0.046003445982933044, 0.038813453167676926, -0.04336702078580856, -0.01935798116028309, 0.10351325571537018, 0.004134123679250479, 0.05963321402668953, 0.09394298493862152, -0.013809329830110073, -0.013552077114582062, 0.04177863523364067, -0.08776630461215973, -0.0065488009713590145, 0.10206349194049835, -0.04522090032696724, -3.623375960160047e-05, 0.045103222131729126, 0.052669260650873184, 0.06780340522527695, 0.04591989144682884, 0.020655466243624687, 0.0047663431614637375, -0.010030149482190609, -0.07247824966907501, -0.06608963757753372, 0.1054053083062172, 0.04943486675620079, -0.002295510843396187, 0.042054127901792526, -0.0008162901503965259, 0.024956749752163887, -0.07941175252199173, 0.03797028958797455, -0.05419785529375076, 0.12010286748409271, -0.034767817705869675, -0.07037504762411118, -0.10565382987260818, 0.031773921102285385, -0.009501802735030651, -0.028966868296265602, 0.03203503414988518, 0.003887782571837306, -0.03991207480430603, -0.10875225067138672, -0.014666512608528137, -0.013807251118123531, -0.05857950448989868, -0.03199707716703415, -0.010555311106145382, -0.0360867977142334, -0.04628930240869522, -0.058514878153800964, -0.04022002965211868, -0.08956266194581985, 0.021358052268624306, 0.04974709078669548, -0.027608703821897507, 0.03585701435804367, 0.00013480489724315703, 0.04530297592282295, -0.0261716078966856, 0.07085743546485901, -0.00585220567882061, 0.10253677517175674, 0.11111430078744888, -0.04178132861852646, -0.10344159603118896, 0.0025326856411993504, -0.08897490054368973, -0.12312164157629013, 0.13051997125148773, 0.0599413700401783, -0.018799427896738052, 0.012205122038722038, -0.0608062781393528, 0.05002424865961075, 0.03608064725995064, 0.01815839670598507, 0.05926843732595444, 0.03671729937195778, -0.061866648495197296, 0.03005032055079937, 0.07277130335569382, -0.01761906035244465, 0.056138861924409866, 0.07874082028865814, 0.023652605712413788, -0.058808110654354095, 0.010394986718893051, 0.006774437613785267, 0.011099446564912796, -0.08149227499961853, -0.0390697717666626, -0.015675559639930725, 0.01753135398030281, 0.08681045472621918, -0.02147284522652626, 0.023680830374360085, 0.05146811157464981, 0.0530802384018898, -0.019403835758566856, -0.03276534005999565, 0.07825887203216553, -0.09225612878799438, -0.020795587450265884, 0.03312436863780022, -0.006758431438356638, -0.06973984092473984, -0.015889907255768776, -0.09550196677446365, 0.047655753791332245, -0.027302252128720284, 0.06032063439488411, 0.046395622193813324, -0.0622817799448967, -0.040248848497867584, 0.043754734098911285, 0.04508819058537483, 0.03521722927689552, -0.06534698605537415, 0.04979763180017471, 0.03634810447692871, -0.07880987226963043, -0.0777592882514, 0.09980916231870651, 0.04316028580069542, -0.02121780999004841, -0.06275785714387894, 0.0579506941139698, -0.06239938363432884, -0.0279177725315094, 0.05495947599411011, 0.07243388146162033, 0.04384813830256462, -0.01497808750718832, 0.06350632756948471, 0.025382429361343384, -0.03355718404054642, 0.004226658493280411, 0.09727304428815842, -0.011996583081781864, 0.04277952015399933, 0.03234495222568512, -0.060480084270238876, -0.0646071657538414, -0.05874672159552574, -0.01723526045680046, -0.04461824521422386, 0.07288656383752823, -0.02918131649494171, -0.05206560343503952, 0.07608459144830704, 0.03462369367480278, -0.04839838668704033, 0.0047079818323254585, 0.04674193635582924, -0.040775835514068604, 0.08742013573646545, 0.018547702580690384, 0.05520891770720482, 0.02637557126581669, -0.08460145443677902, -0.014590153470635414, 0.0010717773111537099, -0.06633113324642181, 0.01879284903407097, -0.0288500115275383, -0.0916203111410141, 0.03737206757068634, 0.015702152624726295, 0.019460491836071014, 0.056895967572927475, 0.07665861397981644, -0.018621550872921944, -0.08039447665214539, 0.06737394630908966, -0.03060626983642578, -0.02185741625726223, -0.011985968798398972, 0.04436672106385231, 0.08633138239383698, -0.01743338257074356, 0.08285678178071976, 0.043557748198509216, 0.0596538744866848, 0.07161473482847214, -0.03697231411933899, -0.06287916749715805, -0.07334146648645401, 0.052651140838861465, -0.0344177782535553, -0.023115966469049454, 0.04967140033841133, 0.0833548903465271, 0.06153363734483719, 0.053685449063777924, 0.04554092511534691, 0.029235590249300003, 0.016746550798416138, 0.07164346426725388, -0.055059608072042465, 0.07320073246955872, 0.04065577685832977, -0.07843788713216782, 0.052260350435972214, -0.03844267502427101, 0.07785554975271225, -0.06339479237794876, 0.07095077633857727, -0.022707095369696617, 0.025123819708824158, -0.08337358385324478, 0.03986988589167595, 0.054369136691093445, -0.025003790855407715, 0.044906750321388245, -0.015598660334944725, 0.07934755086898804, 0.0048805586993694305, 0.003943064250051975, -0.07122282683849335, 0.09769245237112045, 0.00674104131758213, 0.06104721128940582, -0.11941520869731903, 0.049593236297369, -0.04783942550420761, -0.02941150963306427, 0.043465275317430496, 0.08031179755926132, 0.06420350074768066, -0.00819314643740654, 0.029843082651495934, 0.07830512523651123, -0.0828026533126831, -0.13123197853565216, -0.09060037136077881, -0.0190332792699337, 0.07765224575996399, -0.05363977700471878, 0.025459272786974907, 0.056177135556936264, -0.0037162057124078274, 0.019187508150935173, -0.000818574451841414, -0.034386295825242996, -0.04458155483007431, 0.0639963150024414, 0.06752461194992065, -0.01578027568757534, 0.03774264454841614, -0.07133626937866211, 0.05119939148426056, -0.04664082080125809, 0.015709713101387024, -0.030100107192993164, 0.044679440557956696, -0.09470055997371674, -0.061832014471292496, 0.044645607471466064, -0.05697469040751457, -0.013922290876507759, 0.029452983289957047, 0.009829787537455559, 0.04748453199863434, 0.037629384547472, -0.06246321275830269, -0.02081645280122757, 0.05818265676498413, 0.0302920900285244, -0.07552716881036758, 0.030893228948116302, -0.014229674823582172, -0.05248645693063736, 0.040181964635849, -0.00532769737765193, -0.05325393006205559, 0.04932795464992523, -0.05993807315826416, 0.052325084805488586, -0.05007384717464447, 0.04025345668196678, 0.05790401250123978, -0.05105256661772728, -0.06684742867946625, 0.0677696168422699, 0.049341779202222824, -0.01245140377432108, 0.05649570748209953, 0.0858982652425766, -0.06781882792711258, -0.017181171104311943, -0.03586903214454651, -0.013630869798362255, 0.02725783921778202, -0.10035378485918045, -0.0402519628405571, 0.11289285868406296, -0.09211887419223785, 0.03390146791934967, -0.0481342114508152, -0.011501704342663288, 0.08341207355260849, 0.07414768636226654, 0.021879659965634346, 0.02131129428744316, -0.017186300829052925, -0.03574724495410919, -0.07045973837375641, -0.02348671294748783, -0.022082384675741196, -0.05603531375527382, -0.008631641045212746, 0.008957966230809689, 0.06707487255334854, 0.014787332154810429, -0.07094781845808029, 0.12378522753715515, 0.031414009630680084, 0.024036983028054237, 0.09855176508426666, -0.07640400528907776, 0.050385769456624985, -0.007575146853923798, 0.04835375398397446, 0.010574842803180218, 0.03262542933225632, -0.06701786071062088, -0.07041925936937332, -0.09388613700866699, -0.04875427857041359, -0.000451105268439278, -0.007405001204460859, 0.05886000022292137, 0.00869574025273323, -0.02569820173084736, 0.03210313245654106, -0.06784583628177643, -0.014176723547279835, -0.04802151769399643, -0.07313898205757141, -0.01578521728515625, 0.029135752469301224, -0.09402511268854141, 0.05355890095233917, 0.02927997335791588, 0.0396522618830204, 0.01914301887154579, 0.04541520029306412, -0.06107356771826744, 0.04181297495961189, 0.030222037807106972, -0.08508674055337906, 0.06551270931959152, 0.0014703968772664666, 0.0445844940841198, 0.017075061798095703, 0.08327089995145798, 0.010547850281000137, 0.019324755296111107, -0.08234809339046478, -0.03247160464525223, 0.05809418112039566, 0.04716288298368454, -0.07435473054647446, -0.03592367470264435, -0.02207573503255844, 0.06050467491149902, -0.020130988210439682, 0.040266599506139755, 0.05918310955166817, -0.04632113128900528, 0.062028151005506516, 0.027781784534454346, -0.07947760075330734, -0.0753868892788887, -0.08523637056350708, 0.005652015097439289, 0.008122386410832405, -0.07185713201761246, -0.05500204116106033, -0.05142436921596527, 0.06377846747636795, 0.09142731875181198, -0.04088100790977478, -0.005060066934674978, -0.08850885927677155, -0.0671200305223465, 0.08686061948537827, -0.05039861798286438, 0.026305533945560455, 0.038161903619766235, -0.004011253360658884, -0.038660962134599686, -0.10388866811990738, -0.03706509247422218, -0.09509319067001343, -0.03445808216929436, 0.028513699769973755, 0.014430271461606026, -0.048501189798116684, -0.08167997002601624, -0.014022458344697952, -0.08350653201341629, -0.040008433163166046, 0.008415011689066887, -0.009893512353301048, -0.0055380635894834995, -0.03365971893072128, 0.009495355188846588, 0.0733954906463623, -0.08366578817367554, 0.05509395897388458, -0.06515772640705109, 0.07667011022567749, -0.036087922751903534, 0.014733508229255676, -0.03783421218395233, -0.06909128278493881, -0.06399821490049362, -0.05587233230471611, -0.009580808691680431, 0.02961445041000843, -0.05658169835805893, 0.040641576051712036, -0.02037552185356617, -0.08835111558437347, -0.04043450951576233, 0.030330240726470947, -0.05345022305846214, 0.05633588135242462, 0.038377463817596436, -0.03740450367331505, 0.061425112187862396, -0.05570603162050247, -0.007580092176795006, -0.08040662109851837, 0.03294563665986061, -0.0677318274974823, -0.054051682353019714, 0.0001500981888966635, -0.07111098617315292, 0.00043313278001733124, -0.017119042575359344, -0.09901288151741028, 0.033437930047512054, -0.021132048219442368, -0.02410484477877617, 0.062180038541555405, -0.05954602360725403, -0.04554285481572151, 0.06670410931110382, 0.03811224550008774, -0.03769279643893242, -0.03208872675895691, 0.06425776332616806, 0.0016379000153392553, 0.0053016431629657745, 0.03869479522109032, -0.02426219917833805, 0.029344910755753517, -0.0038994471542537212, 0.023166602477431297, -0.025120152160525322, 0.04935356602072716, -0.09666295349597931, -0.09397472441196442, 0.052882011979818344, -0.05407247319817543, -0.05644276738166809, 0.04701949656009674, 0.08964811265468597, 0.10488763451576233, 0.08596797287464142, -0.04658712446689606, -0.07663051038980484, -0.0075147198513150215, -0.018039099872112274, 0.1046205684542656, 0.05431496351957321, -0.03902754560112953, -0.0008014453342184424, -0.02381378971040249, -0.055525004863739014, 0.009872740134596825, -0.011676180176436901, 0.047183357179164886, -0.04740263149142265, -0.04753721505403519, -0.05441250279545784, 0.01343583408743143, -0.07099670171737671, 0.10563495755195618, 0.10646254569292068, 0.04574218764901161, 0.060553617775440216, 0.014991638250648975, 0.03242878615856171, 0.039239685982465744, 6.35463948128745e-05, 0.055741775780916214, -0.020763901993632317, 0.003608561120927334, -0.03896605595946312, 0.08446351438760757, -0.013506139628589153, -0.006147816311568022, 0.0001505560358054936, -0.015217525884509087, 0.05121159926056862, 0.04524482041597366, -0.0992126539349556, -0.06217120960354805, -0.024949295446276665, 0.05219556763768196, -0.047560323029756546, 0.057944752275943756, -0.030178463086485863, 0.0005667569348588586, -0.09846484661102295, -0.09011334180831909, 0.0647682398557663, 0.0397697314620018, 0.08319855481386185, -0.058111097663640976, -0.04775189608335495, -0.009796383790671825, 0.05777069926261902, 0.006346333306282759, 0.022253872826695442, 0.05402722209692001, 0.00874091312289238, 0.05769350007176399, -0.0759936198592186, 0.03462766110897064, 0.028187470510601997, 0.0873664990067482, -0.027428757399320602, -0.10323632508516312, 0.09794613718986511, 0.08291499316692352, 0.03254687786102295, -0.01965918391942978, 0.06022382900118828, 0.09832806885242462, -0.05513225123286247, -0.06112626567482948, -0.02006182260811329, 0.0884082242846489, 0.04206118360161781, -0.08579721301794052, 0.02783406525850296, -0.034856826066970825, 0.0076951924711465836, 0.09991738945245743, -0.06021900847554207, -0.0781833603978157, -0.08733727782964706, 0.018955495208501816, -0.07553897798061371, -0.027954550459980965, 0.0631372332572937, 0.005972692742943764, 0.06683224439620972, 0.019558170810341835, -0.10412184149026871, -0.06046498566865921, 0.007659193594008684, 0.02594289742410183, -0.021119026467204094, 0.03978274017572403, -0.001901996205560863, 0.10674522072076797, -0.006875412072986364, -0.034271489828825, -9.695303742773831e-05, 0.017619267106056213, -0.010377177968621254, -0.07736868411302567, -0.10558924078941345, -0.003119352739304304, 0.0025500343181192875, 0.05246010050177574, 0.0868363156914711, -0.012295248918235302, 0.02758265845477581, -0.04900004714727402, 0.0045950934290885925, -0.04989057406783104, 0.03799216449260712, -0.07956359535455704, 0.006271203514188528, -0.07235733419656754, -0.02385438047349453, -0.0480298176407814, -0.018526436761021614, 0.02745780348777771, 0.03797326609492302, -0.05245354399085045, -0.027801547199487686, -0.04593916982412338, -0.012328249402344227, -0.03540460392832756, 0.06666502356529236, 0.043173108249902725, 0.041570521891117096, 0.03892369940876961, 0.01632482372224331, 0.004880372900515795, 0.05057992413640022, -0.0014197072014212608, 0.09045261144638062, -0.06831870973110199, -0.017724227160215378, 0.04078643023967743, -0.07466830313205719, -0.07532201707363129, -0.014376004226505756, 0.08573409169912338, -0.08010358363389969, -0.07098694890737534, -0.10209628939628601, 0.04418753460049629, 0.05432623624801636, -0.09900524467229843, 0.0778484120965004, 0.02296597696840763, -0.00986978504806757, 0.08051785081624985, 0.013413324020802975, -0.02203904092311859, -0.018410252407193184, 0.02910604141652584, 0.0929993987083435, 0.04158717021346092, 0.011050727218389511, -0.045239757746458054, 0.015493149869143963, -0.03571131452918053, -0.048858802765607834, -0.053582701832056046, -0.013034142553806305, -0.01996377669274807, -0.09716109186410904, -0.0034951798152178526, -0.07035624980926514, -0.05882030352950096, 0.03487840294837952, -0.051778241991996765, -0.04394562169909477, -0.048351436853408813, -0.021165456622838974, -0.02126285620033741, 0.06571578979492188, -0.007732823956757784, -0.03038359433412552, 0.024583933874964714, -0.05936266481876373, -0.08283479511737823, 0.06192026287317276, 0.0362636037170887, 0.0642860010266304, -0.055899906903505325, 0.05069540813565254, -0.0653524398803711, 0.0734458863735199, 0.05120674520730972, 0.015246202237904072, 0.04575297608971596, 0.06393417716026306, 0.08801797032356262, -0.06747742742300034, 0.05117672309279442, 0.028333548456430435, -0.011043231934309006, 0.04713810607790947, 0.023286400362849236, 0.08292210102081299, -0.038953982293605804, 0.007111409213393927, -0.013863054104149342, -0.0923445001244545, -0.045745737850666046, 0.03673715889453888, -0.004225387237966061, -0.007869279943406582, -0.005898940842598677, 0.1166907399892807, -0.058875102549791336, 0.08440426737070084, -0.0027686250396072865, -0.053793489933013916, 0.007407134398818016, -0.05797288939356804, -0.011589366011321545, 0.04925825074315071, 0.006229580380022526, 0.061125658452510834, 0.049536652863025665, -0.038444679230451584, 0.015408657491207123, 0.009543977677822113, 0.015323868952691555, 0.03145664557814598, 0.033163752406835556, -0.122923843562603, 0.024129977449774742, 0.021293068304657936, 0.04896874353289604, 0.09679021686315536, 0.024518070742487907, -0.004917691927403212, -0.011103675700724125, -0.090556301176548, -0.0465279296040535, -0.015000438317656517, -0.062028393149375916, -0.06403503566980362, -0.04524553567171097, -0.029275501146912575, 0.03082987666130066, 0.03879015892744064, -0.029138868674635887, -0.09459096193313599, -0.04293997585773468, -0.032953549176454544, 0.06446108222007751, 0.005835468880832195, 0.007496806792914867, 0.055121876299381256, -0.06423301994800568, 0.060917798429727554, 0.0724588930606842, 0.00735100731253624, 0.05045325681567192, -0.0359007827937603, 0.09143747389316559, 0.01641467772424221, -0.012835393659770489, -0.018763747066259384, 0.018359137699007988, 0.0727597177028656, -0.0618223175406456, 0.012863731943070889, -0.043136365711688995, 0.0015975056448951364, -0.0391237698495388, -0.052849914878606796, 0.04878288507461548, 0.013675929047167301, 0.054103270173072815, 0.056314609944820404, 0.07494699209928513, 0.05523622781038284, -0.04509704187512398, -0.04749017581343651, -0.0730387270450592, 0.043489422649145126, -0.019688157364726067, 0.08476763218641281, 0.08403415232896805, 0.06980166584253311, -0.06902307271957397, -0.07323862612247467, -0.04318242520093918, -0.07861780375242233, 0.03527668118476868, -0.046277839690446854, -0.04692019149661064, 0.07440859079360962, 0.033003922551870346, 0.06745851784944534, 0.02037934586405754, -0.07249992340803146, 0.04480172321200371, 0.020754722878336906, -0.001201288541778922, -0.010378818958997726, -0.022477656602859497, 0.07981535792350769, -0.05687442049384117, -0.03820826858282089, -0.01166408509016037, -0.070759616792202, -0.02791820652782917, -0.015230527147650719, 0.025236574932932854, 0.07586808502674103, -0.06001577526330948, -0.06032274663448334, 0.02670609951019287, 0.005109019111841917, 0.00979806948453188, 0.032200898975133896, -0.05391964316368103, -0.053015731275081635, 0.08879879117012024, -0.08647733926773071, -0.0815562903881073, -0.016730010509490967, -0.017754247412085533, -0.007466540206223726, -0.004810141399502754, 0.041805338114500046, 0.004372915718704462, 0.01181792188435793, -0.061455827206373215, 0.044897232204675674, 0.05874873697757721, -0.0041781081818044186, 0.07254257798194885, -0.05865704268217087, -0.014703316614031792, 0.03713337704539299, 0.0018589816754683852, -0.02700168825685978, 0.045139700174331665, -0.05951963737607002, 0.011847488582134247, 0.008082975633442402, 0.04861743003129959, -0.06122615560889244, -0.034247010946273804, -0.005665700417011976, -0.045489344745874405, -0.07044000178575516, -0.00032836225000210106, -0.08948948234319687, 0.097109355032444, -0.038422148674726486, 0.11128400266170502, 0.10415363311767578, 0.05279824137687683, -0.00031170863076113164, 0.08393091708421707, 0.08374495059251785, 0.05009052902460098, 0.023601025342941284, 0.05751436948776245, -0.06251810491085052, -0.0056579881347715855, -0.024321235716342926, -0.03975171595811844, -0.06731303036212921, 0.047380272299051285, 0.07110889256000519, -0.003940612077713013, -0.06214139237999916, 0.08065397292375565, 0.0007749742944724858, -0.023216797038912773, 0.028772423043847084, -0.0011805080575868487, -0.059947285801172256, -0.016716351732611656, -0.041103485971689224, 0.0364837646484375, 0.08534728735685349, -0.02203497663140297, -0.02661004289984703, -0.07583227753639221, 0.05639931932091713, 0.047515660524368286, 0.06701871752738953, 0.05725240334868431, -0.08580824732780457, -0.009265431202948093, -0.005659239366650581, -0.015506850555539131, 0.03499295189976692, -0.059553954750299454, 0.011860049329698086, 0.05318151414394379, -0.006613207049667835, -0.06476327776908875, -0.06411631405353546, -0.012270243838429451, -0.018816187977790833, 0.04418698698282242, 0.029239369556307793, -0.023809244856238365, -0.04671448841691017, 0.05400049686431885, -0.07571206241846085, 0.0396093986928463, -0.013571026735007763, 0.07130323350429535, 0.06788738816976547, 0.043377939611673355, 0.06997031718492508, -0.012466298416256905, -0.023351717740297318, 0.08781962841749191, 0.0804223120212555, 0.034676115959882736, -0.0005856475909240544, 0.05735703930258751, 0.102533720433712, -0.08474524319171906, 0.06442253291606903, -0.060381900519132614, -0.010077000595629215, 0.04845372959971428, -0.04776516929268837, 0.04784448817372322, 0.029607927426695824, -0.09144587814807892, -0.034188251942396164, 0.06862210482358932, 0.07463095337152481, 0.08760372549295425, 0.03733124956488609, 0.05606880784034729, 0.018444860354065895, -0.018591383472085, -0.006690239999443293, -0.03254485875368118, 0.010823548771440983, 0.059792302548885345, 0.11833033710718155, 0.10201244056224823, -0.029460128396749496, -0.02227484993636608, -0.039644647389650345, 0.07137181609869003, 0.03851917013525963, -0.001850472646765411, -0.005420385394245386, 0.09092021733522415, -0.02265806868672371, 0.05446215346455574, -0.09567024558782578, -0.06073300912976265, 0.040728673338890076, 0.06366642564535141, 0.07640082389116287, 0.08603662997484207, -0.009819524362683296, -0.04831426218152046, 0.04681016877293587, 0.006491813343018293, -0.035092271864414215, -0.03747749328613281, 0.009238801896572113, -0.05551757663488388, 0.059978511184453964, -0.03734072670340538, 0.07936842739582062, 0.013518591411411762, -0.08513008803129196, -0.07095101475715637, -0.0172344371676445, 0.042616669088602066, 0.004911133553832769, -0.03047754429280758, -9.083544136956334e-05, 0.05881555750966072, 0.019319329410791397, 0.015156029723584652, 0.07754135876893997, 0.036660853773355484, -0.05293860286474228, 0.0070745209231972694, -0.11237775534391403, 0.03655269369482994, 0.05253272131085396, -0.0030954061076045036, 0.016436288133263588, -0.07181401550769806, -0.018653234466910362, 0.021127445623278618, 0.052761998027563095, -0.06810150295495987, 0.08768343925476074, -0.029272207990288734, -0.03311099484562874, -0.008759919553995132, 0.04634427651762962, 0.017568396404385567, -0.002268181648105383, -0.10243481397628784, -0.07585632055997849, 0.014752035029232502, -0.010718610137701035, -0.021777717396616936, -0.03082805499434471, 0.029634060338139534, -0.07706979662179947, -0.06542308628559113, 0.019402876496315002, 0.06539861112833023, -0.034421034157276154, -0.06341362744569778, -0.017434636130928993, -0.026465216651558876, -0.04743087664246559, -0.07670959830284119, -0.08079972118139267, -0.10670658946037292, -0.005400367546826601, -0.060506585985422134, -0.011326560750603676, -0.10908935219049454, -0.06128388270735741, 0.06398529559373856, -0.09443484246730804, -0.010484011843800545, 0.05791005119681358, 0.034170493483543396, 0.021646391600370407, 0.0038040694780647755, -0.0525226816534996, 0.03686884790658951, -0.03469577431678772, -0.01883707381784916, 0.024656731635332108, -0.031362153589725494, -0.03541971370577812, 0.003955715335905552, -0.02449517510831356, -0.03209628164768219, -0.08376861363649368, -0.017751211300492287, -0.008562825620174408, 0.07709700614213943, -0.0073962463065981865, -0.09228730946779251, -0.07045243680477142, 0.05371810123324394, 0.03245401009917259, -0.0653868094086647, 0.048923514783382416, -0.05597180500626564, -0.05488290265202522, -0.07400044053792953, 0.12842853367328644, 0.009297466836869717, 0.032465655356645584, -0.0009890380315482616, -0.01653820089995861, -0.07102399319410324, 0.05531895160675049, 0.017891163006424904, 0.05318880453705788, 0.09217432141304016, -0.09135770052671432, 0.017877481877803802, -0.06554502248764038, 0.06186940148472786, 0.032971784472465515, -0.07407450675964355, -0.02482917346060276, 0.041598882526159286, -0.07137841731309891, -0.043858353048563004, 0.022162916138768196, -0.07672704011201859, -0.01721733808517456, -0.045345719903707504, -0.0049703605473041534, 0.03244403377175331, -0.009863344021141529, 0.040267329663038254, 0.02326922118663788, 0.030241860076785088, -0.03500699996948242, -0.03019355796277523, 0.05209658667445183, 0.053037382662296295, -0.009710616432130337, -0.02497045136988163, 0.03343988209962845, 0.025929881259799004, -0.12495020776987076, -0.009974041022360325, 0.03026464767754078, 0.01188631821423769, 0.006542118266224861, -0.03517982363700867, 0.07988031953573227, -0.04442465677857399, 0.0514955036342144, -0.0753791406750679, 0.0047273035161197186, 0.11291714757680893, 0.029074225574731827, -0.08905481547117233, -0.014868481084704399, -0.03723359853029251, 0.04867557808756828, 0.05055742338299751, -0.001733160112053156, 0.039560046046972275, 0.06561266630887985, 0.0614590123295784, -0.06829171627759933, 0.062040604650974274, -0.016217399388551712, -0.03584057092666626, 0.03993651270866394, -0.03925851359963417, 0.030488261952996254, 0.061308953911066055, -0.04301396384835243, -0.004166097845882177, -0.13644471764564514, -0.07744601368904114, -0.04853540658950806, -0.0765511691570282, 0.07335703074932098, -0.0774930790066719, -0.005814362317323685, 0.02750561572611332, 0.026592601090669632, -0.03910618647933006, -0.0030212493147701025, 0.06708813458681107, -0.03888370096683502, -0.1423112154006958, 0.03813845291733742, 0.002267837058752775, -0.07477087527513504, 0.06730122119188309, 0.005303986370563507, 0.05210781842470169, 0.012409141287207603, 0.099240742623806, 0.007096602115780115, -0.019717244431376457, -0.023382289335131645, -0.07027922570705414, -0.07934698462486267, -0.07957093417644501, 0.015122851356863976, -0.0927194207906723, -0.018652554601430893, -0.026234321296215057, 0.09453876316547394, 0.11404721438884735, 0.04987223818898201, -0.03372286632657051, 0.07172571867704391, -0.007015710696578026, -0.05654701590538025, -0.04964020848274231, 0.007498492486774921, -0.0069012269377708435, -0.054492488503456116, 0.017287740483880043, -0.03911907225847244, -0.02346045896410942, -0.05766678974032402, 0.0705208107829094, -0.08224619925022125, -0.11812300235033035, 0.038116034120321274, -0.022554541006684303, -0.10281744599342346, -0.02123899571597576, -0.0920456126332283, 0.05720250681042671, 0.04697795584797859, 0.10232878476381302, -0.08485504984855652, -0.09646201878786087, -0.0444999635219574, -0.07690147310495377, 0.022530492395162582, 0.03845077008008957, 0.05124197155237198, -0.11976730078458786, 0.04000287503004074, -0.08173635601997375, 0.0007213180651888251, -0.022163869813084602, 0.04238174855709076, 0.07721596211194992, -0.057787224650382996, 0.0014615283580496907, -0.07881292700767517, 0.03500709310173988, 0.025663023814558983, -0.08087633550167084, 0.004455905873328447, -0.026605777442455292, 0.045952484011650085, -0.08802811056375504, -0.0034551124554127455, -0.03986813500523567, 0.002129958476871252, 0.0327579565346241, -0.04255114495754242, -0.04081573709845543, -0.0772894099354744, -0.10160531103610992, -0.05164157971739769, 0.007401586975902319, 0.04805221036076546, -0.029493948444724083, -0.013905934989452362, -0.04621893912553787, 0.013637105002999306, 0.07534689456224442, -0.003210838884115219, 0.05784326046705246, 0.07866893708705902, 0.06619183719158173, -0.04930568486452103, 0.013572324067354202, -0.06050388514995575, -0.09252118319272995, 0.06546758115291595, -0.0465712696313858, 0.03475360572338104, 0.055603235960006714, 0.07050441205501556, -0.01932601071894169, 0.03343689814209938, -0.08096808195114136, 0.01636314205825329, 0.009059730917215347, -0.04217499494552612, 0.041135210543870926, -0.04441118240356445, 0.03225669637322426, -0.08496859669685364, -0.01037164218723774, -0.11557578295469284, 0.10907264053821564, -0.03858908265829086, 0.03270411118865013, 0.02808089554309845, -0.00674000708386302, 0.01702233962714672, 0.08240959793329239, -0.09909391403198242, 0.01267148181796074, -0.1025841161608696, 0.00944371335208416, 0.07024285197257996, -0.04531587287783623, 0.06255600601434708, 0.011836301535367966, 0.08648335933685303, -0.05636619031429291, -0.01628495194017887, 0.05763205140829086, 0.03285839036107063, 0.09217189252376556, -0.03929721936583519, 0.010834060609340668, -0.008706197142601013, -0.013858099468052387, 0.05312643572688103, 0.018659457564353943, -0.006362284999340773, -0.047092147171497345, -0.06190834194421768, 0.10099530965089798, 0.000792419770732522, -0.0032810382544994354, -0.06378929316997528, 0.06259970366954803, 0.018935777246952057, 0.02290172316133976, 0.07358264923095703, 0.062112994492053986, 0.05037902668118477, -0.0196385458111763, 0.009339757263660431, 0.024475673213601112, -0.06566445529460907, 0.01305370219051838, -0.08583983778953552, 0.048465121537446976, 0.0015510199591517448, -0.02441391721367836, 0.002113357651978731, 0.08172006160020828, 0.076353058218956, 0.09456611424684525, -0.058858390897512436, -0.04740586876869202, 0.017328573390841484, 0.06890389323234558, 0.0978061780333519, -0.048172302544116974, 0.08387569338083267, -0.04621032997965813, 0.01537271961569786, -0.016187205910682678, -0.07223773747682571, 0.02722650021314621, 0.03911028429865837, -0.028264831751585007, 0.05836539342999458, -0.10634946078062057, 0.084417425096035, 0.08038512617349625, -0.07444419711828232, 0.044361915439367294, -0.035969674587249756, -0.03974176198244095, 0.08886511623859406, 0.05901968479156494, -0.032090410590171814, -0.11204138398170471, 0.013367516919970512, -0.010621796362102032, -0.006955320946872234, 0.09973432123661041, -0.02948945388197899, 0.034875866025686264, 0.033159226179122925, -0.019200017675757408, 0.04224724322557449, -0.02088186889886856, 0.019543340429663658, 0.028978554531931877, 0.03339822590351105, -0.013693341054022312, 0.028407208621501923, -0.07655918598175049, -0.027021916583180428, 0.062119849026203156, -0.01806054823100567, 0.05065721273422241, 0.06821302324533463, -0.07275443524122238, 0.03983950987458229, -0.013995886780321598, -0.03566692769527435, -0.008227076381444931, -0.08343682438135147, 0.01812675967812538, -0.0432477630674839, -0.03766647353768349, 0.028839953243732452, -0.08425738662481308, -0.055363163352012634, -0.04007811099290848, 0.10082312673330307, 0.06776061654090881, 0.10052777826786041, -0.057132646441459656, 0.027393341064453125, 0.07804343849420547, -0.09796341508626938, 0.011372716166079044, -0.03224605694413185, 0.07563045620918274, 0.060209427028894424, 0.022493990138173103, 0.06714483350515366, 0.07616830617189407, 0.028630202636122704, -0.03687618300318718, -0.053775958716869354, -0.04342789575457573, 0.0463954396545887, 0.02568628638982773, -0.012307250872254372, -0.02410372905433178, -0.07981821894645691, -0.09585487097501755, -0.011951589025557041, 0.07116580754518509, 0.08233650028705597, 0.050018757581710815, -0.022982096299529076, 0.0810055360198021, 0.026149921119213104, 0.06349464505910873, 0.029330803081393242, -0.01882772706449032, -0.0519598089158535, -0.018776554614305496, -0.08451198786497116, -0.07008351385593414, -0.04869808256626129, -0.04477404057979584, -0.09382416307926178, 0.09088126569986343, 0.05167435109615326, -0.06435422599315643, -0.027422815561294556, -0.002101161051541567, -0.08731935918331146, -0.00768183683976531, 0.019123747944831848, 0.017538581043481827, -0.04728619009256363, -0.10031630843877792, 0.046630192548036575, -0.03939557075500488, 0.013782734051346779, 0.046456482261419296, -0.07743293792009354, -0.01210552267730236, 0.001073002815246582, 0.010031533427536488, 0.00458144536241889, -0.08142010122537613, -0.005111848935484886, 0.05781777203083038, -0.043402425944805145, -0.003613711567595601, 0.017022956162691116, -0.005399737507104874, 0.023760458454489708, -0.030148494988679886, -0.03666341304779053, 0.04324799403548241, -0.0706113800406456, -0.06245718523859978, 0.003546972991898656, 0.068540059030056, 0.03267347067594528, -0.014898029156029224, -0.07438534498214722, -0.0738464742898941, -0.014324466697871685, -0.05006586015224457, -0.05562254413962364, -0.09277866780757904, 0.01979343220591545, -0.04746605455875397, 0.050938285887241364, 0.08333571255207062, 0.024664537981152534, 0.05343430116772652, -0.07211267203092575, 0.005801772233098745, 0.07613037526607513, 0.026423532515764236, 0.002705186139792204, 0.013432683423161507, 0.042916957288980484, 0.047646865248680115, -0.04208577796816826, 0.010160799138247967, -0.036585986614227295, 0.0902651995420456, -0.10250237584114075, 0.07496427744626999, -0.07663267850875854, 0.009001399390399456, -0.05914559215307236, 0.01892414689064026, -0.019662559032440186, -0.08155972510576248, 0.07175526767969131, 0.03702196106314659, 0.1029098704457283, -0.032829005271196365, 0.05828740820288658, 0.03777804225683212, -0.01675010286271572, 0.04501134529709816, 0.045778632164001465, -0.025378474965691566, 0.0358513742685318, 0.04039726033806801, 0.04311271011829376, -0.0073592993430793285, -0.05962807312607765, -0.07033893465995789, 0.04887804016470909, -0.04695510491728783, -0.04593278840184212, -0.05168002098798752, 0.025324424728751183, -0.08583042770624161, 0.017059432342648506, 0.10647516697645187, -0.05234232172369957, -0.020122814923524857, -0.06456036120653152, -0.09744904935359955, -0.0042741200886666775, -0.0707828477025032, 0.027691952884197235, 0.044994160532951355, -0.037608399987220764, 0.0161605104804039, -0.06717351078987122, 0.058401837944984436, -0.009821806102991104, -0.045916154980659485, -0.019062677398324013, 0.044661786407232285, 0.00574397062882781, 0.08106734603643417, -0.014675307087600231, -0.04619733616709709, 0.04527290165424347, -0.014770406298339367, 0.0854433998465538, -0.07699202001094818, -0.003991472534835339, 0.04184630513191223, 0.03721931204199791, -0.047604139894247055, -0.012373997829854488, -0.04732342064380646, -0.007936480455100536, -0.0033634488936513662, -0.06658313423395157, -0.07676216959953308, 0.008749290369451046, -0.016668492928147316, -0.009693962521851063, -0.044828154146671295, -0.011006157845258713, 0.009605204686522484, -0.08315638452768326, 0.032740529626607895, -0.006958888843655586, -0.08247566968202591, 0.037231989204883575, -0.037718236446380615, 0.08026431500911713, -0.055925071239471436, 0.06514701247215271, 0.000386121217161417, -0.017650596797466278, -0.0182317066937685, 0.06350280344486237, -0.10285632312297821, 0.0014392302837222815, 0.06561385095119476, 0.03436282277107239, 0.04518131539225578, -0.030962495133280754, 0.005068489350378513, -0.05451608821749687, 0.04955221340060234, -0.0022228125017136335, -0.04191656410694122, -0.012464474886655807, -0.10672400891780853, -0.008483145385980606, 0.05085571110248566, 0.07204100489616394, 0.000550862867385149, -0.03954920172691345, -0.012702172622084618, 0.07758826017379761, 0.033082373440265656, -0.09595607221126556, -0.08627098053693771, -0.007828897796571255, -0.07177410274744034, -0.0388968288898468, -0.09245826303958893, 0.006134558469057083, -0.04639463126659393, 0.0055678123608231544, -0.04872945323586464, -0.03286990523338318, 0.028656115755438805, -0.0066608223132789135, 0.07962971180677414, -0.1121751070022583, -0.09157223999500275, -0.03740455210208893, -0.05687751993536949, 0.0029580551199615, 0.06889446824789047, 0.024753980338573456, 0.07233622670173645, -0.09497574716806412, 0.040720827877521515, 0.06066550314426422, -0.04347291216254234, 0.052543554455041885, 0.05331796780228615, 0.02009769156575203, -0.0894109234213829, 0.006821147166192532, -0.009929675608873367, 0.0875249058008194, -0.016016699373722076, 0.0818110853433609, 0.006171531043946743, -0.025930633768439293, -0.06312175840139389, -0.06037064269185066, -0.01847747154533863, 0.07515016198158264, 0.010984305292367935, -0.08403127640485764, -0.1021602675318718, 0.048663631081581116, 0.046113576740026474, 0.06302475929260254, -0.09842312335968018, 0.057296041399240494, -0.07466235011816025, 0.07159163057804108, -0.07642849534749985, 0.022167427465319633, -0.07201994955539703, -0.03390730544924736, 0.06737979501485825, 0.04340629279613495, 0.030446315184235573, -0.08329448848962784, -0.05882911756634712, -0.11323945969343185, 0.011559574864804745, -0.0016543719684705138, 0.06349287927150726, -0.024421395733952522, -0.05195498466491699, -0.08988282829523087, -0.09734675288200378, -0.01892448216676712, 0.04971219226717949, 0.03629908710718155, -0.06524783372879028, -0.023661676794290543, 0.07866357266902924, 0.01610119827091694, 0.012467282824218273, 0.038942739367485046, 0.05229131504893303, 0.011798803694546223, -0.08622834086418152, 0.098246730864048, -0.008351108059287071, 0.01073089987039566, -0.05180131644010544, -0.08585628867149353, 0.08443769812583923, 0.032012637704610825, 0.020767180249094963, -0.007538006640970707, 0.0434972383081913, -0.03321586921811104, 0.04760343208909035, -0.009270443581044674, -0.05096714571118355, -0.06282124668359756, -0.10242916643619537, -0.01590409129858017, 0.06385576725006104, -0.032070983201265335, 0.060229308903217316, -0.08095209300518036, -0.027304956689476967, 0.03990006446838379, -0.05822375416755676, 0.014320681802928448, 0.05789050832390785, 0.0009679874638095498, 0.0037143041845411062, -0.04541899263858795, 0.08951094001531601, -0.05072513967752457, 0.09663975983858109, 0.07716741412878036, 0.01105209719389677, -0.05355973169207573, 0.02513064816594124, 0.031070470809936523, 0.03473033010959625, -0.038275644183158875, 0.08506687730550766, -0.032416921108961105, 0.0031979940831661224, -0.034302905201911926, -0.033242691308259964, 0.012527923099696636, 0.06467398256063461, 0.07620628923177719, 0.06393316388130188, 0.0731096938252449, 0.055682048201560974, 0.10051464289426804, -0.09341810643672943, 0.0419137217104435, -0.03916187584400177, -0.07349800318479538, 0.005932343192398548, 0.004946001805365086, 0.012671397998929024, 0.09784611314535141, 0.0027231452986598015, -0.05289200693368912, -0.051772210747003555, 0.054433610290288925, 0.0051488569006323814, -0.049250535666942596, -0.09759621322154999, 0.008575474843382835, 0.08890961110591888, 0.05420706793665886, -0.03481663390994072, 0.002329915761947632, -0.015642337501049042, -0.06563080847263336, 0.06589985638856888, -0.04940319433808327, -0.08975937217473984, 0.09939448535442352, 0.030847130343317986, -0.057349856942892075, -0.03854111582040787, -0.06773170083761215, 0.0506126694381237, 0.05803416296839714, 0.05362270027399063, -0.04613058641552925, -0.0481155626475811, 0.1062379777431488, -0.038899488747119904, 0.015278888866305351, -0.08585434406995773, 0.05097101256251335, -0.058919962495565414, 0.01998988352715969, -0.03139601647853851, 0.041129566729068756, 0.028640756383538246, 0.09573440253734589, -0.019907040521502495, 0.0801834836602211, 0.014211312867701054, 0.020385297015309334, -0.012061391957104206, 0.02263552136719227, -0.0052785370498895645, -0.09052769094705582, 0.015290053561329842, 0.10532060265541077, -0.06884299963712692, -0.08246657997369766, -0.06992103159427643, -0.00741103570908308, -0.09819051623344421, 0.09336565434932709, -0.007700260728597641, 0.04458588734269142, -0.02410086803138256, 0.008059024810791016, 0.0702163502573967, -0.028742101043462753, -0.06646193563938141, 0.029368115589022636, -0.06658540666103363, -0.05308042839169502, -0.058524470776319504, -0.07917340844869614, 0.05574030429124832, -0.08860617130994797, 0.006547016557306051, -0.0055426424369215965, 0.05729719251394272, 0.05450174957513809, 0.005846021696925163, 0.018834354355931282, -0.024273378774523735, -0.018516141921281815, -0.005613361019641161, -0.04867545887827873, 0.04559725895524025, 0.010837431997060776, 0.06340370327234268, 0.02125788852572441, -0.07488680630922318, -0.07899825274944305, -0.019467657431960106, -0.018049048259854317, -0.07589028775691986, 0.010604221373796463, -0.04835214465856552, 0.009112982079386711, -0.031172549352049828, -0.04733927547931671, -0.034769050776958466, 0.022862713783979416, -0.04438815265893936, -0.0677344873547554, -0.07181472331285477, 0.05431228131055832, 0.02260349877178669, -0.004248133860528469, 0.07283804565668106, -0.05547402799129486, -0.07833205163478851, 0.04086096212267876, 0.039228253066539764, -0.09352003037929535, -0.04303421080112457, -0.022341595962643623, -0.03740844875574112, 0.0951072946190834, 0.0034107198007404804, -0.07761521637439728, 0.008506184443831444, -0.05738908052444458, 0.0534515455365181, -0.005851454101502895, -0.10274249315261841, -0.04716206714510918, -0.05542292445898056, -0.023642869666218758, -0.013204927556216717, 0.09280552715063095, -0.024497855454683304, 0.0644809752702713, 0.05058363452553749, -0.053754232823848724, -0.005910416599363089, 0.04481310397386551, 0.06546817719936371, 0.0591985359787941, 0.0679042637348175, 0.052660971879959106, 0.01769830286502838, -0.048125971108675, -0.05373736098408699, -0.044547777622938156, 0.0016330779762938619, -0.07102440297603607, -0.01329879555851221, -0.021560458466410637, 0.08348129689693451, -0.029215676710009575, 0.0003739404783118516, -0.04402478411793709, -0.021193748340010643, -0.07463817298412323, -0.050487011671066284, -0.017196863889694214, -0.03912070021033287, 0.053238023072481155, -0.01126587763428688, -0.01764392852783203, -0.022815417498350143, 0.03970976546406746, 0.0371888093650341, -0.024920854717493057, -0.09184008091688156, 0.02240813709795475, 0.0535135418176651, -0.07101020961999893, 0.07161729782819748, -0.002186291152611375, -0.0005990965291857719, 0.052798427641391754, 0.006784582044929266, -0.010746548883616924, 0.05467114970088005, -0.0045399852097034454, -0.004293660633265972, 0.03998340666294098, -0.04556121677160263, -0.03370519354939461, 0.03377075865864754, 0.03974462300539017, -0.024000074714422226, -0.011622917838394642, -0.01803833432495594, 0.04285568371415138, 0.0035563642159104347, 0.01708053983747959, -0.006852447986602783, -0.055285997688770294, 0.03003540448844433, -0.05579972639679909, 0.04871644824743271, -0.08662906289100647, 0.0005774025339633226, -0.032163459807634354, -0.0725722387433052, 0.016914445906877518, 0.012014476582407951, -0.10580679029226303, -0.012259171344339848, -0.07528849691152573, -0.06612791866064072, 0.08629829436540604, -0.01064544077962637, 0.06401149183511734, 0.06447266042232513, 0.09740515798330307, 0.0960582047700882, 0.020482756197452545, -0.022593196481466293, -0.030689027160406113, 0.0868489220738411, 0.05074647068977356, -0.025831688195466995, -0.043184537440538406, -0.007629330735653639, 0.04161032289266586, 0.061864424496889114, 0.02749343030154705, 0.011226516216993332, 0.035138778388500214, 0.015040182508528233, -0.07372202724218369, -0.038532111793756485, -0.07407156378030777, 0.05309076979756355, -0.055759772658348083, -0.03485972434282303, 0.03944618999958038, -0.10376037657260895, -0.05371423438191414, -0.05623307824134827, -0.03841565176844597, -0.04270414263010025, 0.10541173815727234, 0.10145775973796844, -0.021564340218901634, -0.0066091506741940975, 0.03298485279083252, -0.04215332120656967, 0.05307384207844734, 0.07553305476903915, 0.03692245855927467, -0.06881517171859741, -0.024348199367523193, 0.009396902285516262, 0.08569937944412231, 0.03210928663611412, 0.03774711489677429, 0.044981665909290314, 0.008901514112949371, 0.04550328478217125, -0.02710963785648346, -0.031309615820646286, 0.026966698467731476, -0.06411687284708023, 0.11984416097402573, -0.03364512324333191, 0.03869953751564026, 0.0590982623398304, -0.09766571968793869, 0.052443575114011765, -0.06915844231843948, -0.016231345012784004, 0.08309068530797958, -0.05686434730887413, -0.08030863106250763, 0.01380483340471983, 0.013150285929441452, 0.021856315433979034, 0.07231302559375763, 0.08658530563116074, 0.04982272908091545, 0.04338681325316429, 0.01212387252599001, -0.05712876096367836, -0.007416415959596634, -0.038400452584028244, -0.007809313014149666, -0.013995894230902195, -0.009788607247173786, 0.09518539160490036, -0.07732294499874115, -0.01755090244114399, 0.02882484160363674, 0.018949827179312706, -0.0454697459936142, -0.0651707723736763, -0.07376889884471893, 0.06303539872169495, -0.0798487588763237, 0.057589512318372726, -0.10075496137142181, 0.06933415681123734, 0.028833214193582535, 0.08776317536830902, -0.05472743883728981, 0.08340945839881897, -0.034365784376859665, 0.07838833332061768, 0.003865692764520645, -0.02060651034116745, 0.04465455561876297, 0.006265446078032255, -0.04464149475097656, 0.07449287921190262, 0.0037313622888177633, 0.05712855979800224, 0.04582295939326286, -0.03289159759879112, 0.05196099728345871, 0.03165232762694359, -0.036387499421834946, -0.043020378798246384, 0.06072369962930679, -0.05321487411856651, -0.05085166543722153, -0.025908170267939568, -0.04932926222681999, 0.07052995264530182, -0.023305464535951614, 0.013872960582375526, 0.027874642983078957, 0.04975983873009682, 0.05843457207083702, 0.012351014651358128, 0.05062197893857956, 0.03161555901169777, -0.018834585323929787, 0.06930755078792572, 0.06183096393942833, 0.02698812074959278, 0.010947243310511112, -0.024660950526595116, -0.010044575668871403, -0.04613136872649193, 0.07496852427721024, -0.030034217983484268, 0.06223896145820618, -0.041521668434143066, 0.06783430278301239, 0.008796234615147114, -0.03389903903007507, -0.026032883673906326, -0.07006823271512985, -0.022085675969719887, 0.025005487725138664, -0.07470547407865524, -0.05398932471871376, -0.09901444613933563, 0.021824631839990616, -0.046612389385700226, 0.06350784748792648, -0.08609358966350555, 0.058690011501312256, 0.04365954175591469, -0.033496126532554626, 0.005541949067264795, 0.07930704951286316, -0.029182927682995796, -0.05247579887509346, 0.0406128354370594, 0.008643411099910736, 0.0143807427957654, -0.055180974304676056, 0.07619278877973557, -0.04058605059981346, 0.06267564743757248, 0.04548873379826546, -0.05182379111647606, -0.007322343997657299, 0.0593821182847023, 0.0012267074780538678, -0.07030647993087769, 0.06947127729654312, -0.0559709370136261, 0.0035566638689488173, 0.06852168589830399, 0.026577437296509743, -0.019493037834763527, -0.03981168195605278, -0.0530092790722847, -0.048086658120155334, 0.01640697941184044, -0.0990694984793663, -0.0033640938345342875, -0.07767105847597122, -0.02731720171868801, 0.07292412221431732, -0.05802394822239876, 0.07274431735277176, -0.02454143390059471, 0.03980595991015434, 0.03122011013329029, 0.05607921630144119, 0.07380180805921555, -0.05589177832007408, 0.046725936233997345, 0.03183953836560249, -0.07687098532915115, 0.04411394149065018, -0.01258076261729002, 0.029024802148342133, -0.07911279797554016, -0.05752183869481087, 0.022185539826750755, -0.023941971361637115, 0.09735970199108124, 0.04309648647904396, -0.09248887747526169, 0.04239107295870781, 0.017030203714966774, 0.002978515811264515, -0.018531134352087975, -0.052047595381736755, 0.023569222539663315, 0.007668810896575451, 0.030094904825091362, 0.00011261628242209554, 0.03313969075679779, 0.06769891083240509, -0.03752279281616211, -0.025041256099939346, -0.09938100725412369, 0.05508444830775261, -0.04883052036166191, -0.045383572578430176, -0.030235392972826958, -0.03679629787802696, -0.0852029100060463, -0.01750344969332218, 0.023157406598329544, -0.0986725464463234, -0.022054802626371384, 0.0805269256234169, 0.0896918922662735, 0.009565882384777069, 0.0034823790192604065, -0.03194775432348251, -0.02149408683180809, -0.005986499134451151, 0.05991363525390625, -0.050929002463817596, -0.04354505240917206, -0.0546104721724987, 0.04372334107756615, -0.02017705887556076, -0.013266243040561676, -0.00043481774628162384, 0.028043072670698166, 0.053272925317287445, 0.01143477763980627, 0.040296394377946854, 0.021855834871530533, -0.00245311064645648, 0.03019139915704727, -0.050142981112003326, 0.07993905246257782, -0.07090159505605698, -0.017677873373031616, 0.020790092647075653, -0.03330394625663757, -0.01781352609395981, -0.04260970279574394, -0.07283782958984375, -0.07827018946409225, 0.006522332783788443, 0.045960236340761185, 0.07647493481636047, 0.061870135366916656, 0.012078149244189262, -0.004100526683032513, 0.021052787080407143, -0.02340620383620262, -0.012458104640245438, 0.0035749252419918776, -0.05809266120195389, 0.04104044288396835, -0.04961220547556877, -0.09236155450344086, 0.0013927414547652006, 0.060864854604005814, -0.06476827710866928, 0.05831693857908249, -0.0643286406993866, -0.06085342541337013, 0.0528564415872097, -0.035014647990465164, -0.022572550922632217, 0.004319470841437578, -0.07734033465385437, -0.007195994723588228, -0.025141291320323944, -0.0660177618265152, -0.010249990038573742, 0.03508763760328293, -0.0021414891816675663, -0.10091688483953476, -0.09925419092178345, -0.02912835031747818, 0.028705013915896416, 0.06086546555161476, 0.09816987812519073, 0.018020479008555412, 0.046684589236974716, 0.022450769320130348, -0.056775204837322235, 0.02556927502155304, 0.010774053633213043, -0.03971449285745621, 0.025109706446528435, -0.07184093445539474, 0.06636468321084976, -0.028760381042957306, -0.06700856983661652, -0.07198632508516312, -0.05085044726729393, 0.011145264841616154, 0.04713011905550957, -0.01833997666835785, 0.08545580506324768, 0.018546929582953453, -0.040781717747449875, 0.06016077101230621, 0.01129960361868143, 0.06418875604867935, -0.0097882691770792, 0.007843626663088799, -0.012155268341302872, 0.01693917065858841, -0.06407123059034348, 0.02372901514172554, -0.019271353259682655, -0.03456733748316765, -0.036481305956840515, 0.044839926064014435, -0.012008460238575935, -0.016188403591513634, -0.04919161647558212, 0.0935976654291153, 0.032480571419000626, 0.06796558946371078, 0.010109440423548222, -0.10622597485780716, 0.04810032993555069, 0.03684474900364876, 0.03518901392817497, 0.016273560002446175, 0.01134534366428852, 0.040235549211502075, -0.06994082033634186, -0.055208127945661545, -0.04707057774066925, -0.01739044114947319, 0.0033941103611141443, 0.0020750744733959436, 0.08238713443279266, 0.004225180018693209, 0.018645204603672028, 0.0238211527466774, 0.01742144674062729, -0.07102137058973312, 0.06269513070583344, 0.09808703511953354, 0.09098131209611893, -0.05357127636671066, -0.07623137533664703, 0.09818318486213684, 6.417671102099121e-06, 0.04983022063970566, -0.1236482635140419, -0.0789029449224472, -0.010196944698691368, -0.06434153020381927, -0.06832058727741241, -0.06605613976716995, -0.001421600696630776, -0.06972363591194153, -0.05150798708200455, -0.02801446057856083, -0.021222520619630814, 0.05671162158250809, -0.060337331146001816, 0.007961731404066086, -0.050727516412734985, 0.09691332280635834, -0.02580420859158039, 0.053084857761859894, -0.03104150854051113, -0.04253179952502251, 0.008372006006538868, 0.028919856995344162, 0.07065748423337936, -0.03525697812438011, -0.018057681620121002, -0.007468702271580696, -0.0797157734632492, -0.0018294085748493671, -0.010964124463498592, 0.0449996143579483, 0.004017498344182968, 0.0966162234544754, 0.06668642163276672, 0.02127421833574772, -0.059116508811712265, -0.09425459057092667, -0.024819795042276382, 0.05481540411710739, -0.0065439725294709206, 0.06657889485359192, -0.05687815323472023, 0.0643368512392044, 0.025056805461645126, -0.07472915947437286, 0.0708489716053009, -0.013791668228805065, -0.018919894471764565, -0.04528198763728142, 0.06279060244560242, -0.029488593339920044, -0.0024377168156206608, -0.11926427483558655, -0.02350451610982418, -0.01424212846904993, -0.03414090350270271, 0.052898723632097244, 0.004876577295362949, 0.011466202326118946, 0.02867068536579609, -0.03921569511294365, 0.04621488228440285, 0.024756448343396187, -0.01296820305287838, 0.03783244639635086, -0.04860991984605789, -0.07167650759220123, -0.06013098731637001, 0.025587255135178566, -0.01164334174245596, 0.009898807853460312, -0.03034956566989422, 0.05625980719923973, -0.021399972960352898, -0.007116412743926048, 0.060132358223199844, -0.08789533376693726, 0.01315548550337553, -0.05619669333100319, -0.010672361589968204, -0.03714277595281601, 0.02854447439312935, 0.010387872345745564, 0.058357808738946915, 0.07337860763072968, -0.047899167984724045, 0.0022860702592879534, -0.02675558254122734, -0.06950646638870239, -0.04133623093366623, 0.03583141043782234, 0.06991540640592575, 0.017919333651661873, -0.028521627187728882, -0.03727400302886963, 0.02037915773689747, 0.03365814685821533, 0.045700207352638245, 0.06883236020803452, -0.044286537915468216, -0.034051280468702316, 0.0069144428707659245, 0.003940793685615063, 0.06989338248968124, 0.008315632119774818, 0.029464922845363617, 0.07378725707530975, 0.04547981545329094, 0.03617028892040253, -0.05787006393074989, -0.0741247683763504, -0.02774152159690857, -0.09973697364330292, -0.0453488864004612, 0.023489179089665413, -0.03603618964552879, -0.035152535885572433, -0.006021616514772177, -0.04569445177912712, -0.07116646319627762, -0.06640937179327011, -0.014983492903411388, 0.07492489367723465, -0.06861525774002075, -0.046748314052820206, 0.0056892018765211105, 0.00313814589753747, -0.01429071370512247, 0.07722359895706177, -0.01681143045425415, 0.0726642832159996, 0.023677615448832512, 0.08649145066738129, 0.08123219758272171, 0.030417196452617645, -0.07374602556228638, -0.02149476855993271, -0.09644149243831635, -0.006575286854058504, 0.030469316989183426, 0.024173162877559662, -0.024269524961709976, 0.08568596094846725, 0.0911063700914383, 0.08636436611413956, 0.06984459608793259, 0.06753034144639969, -0.0069677443243563175, 0.014035667292773724, 0.043978672474622726, -0.05239066854119301, 0.024743203073740005, -0.010329509153962135, -0.012165834195911884, 0.031002510339021683, -0.0023782099597156048, 0.07082480937242508, -0.10353904217481613, 0.029438160359859467, 0.07567940652370453, 0.0671313926577568, -0.07139190286397934, 0.025492055341601372, 0.0009076603455469012, -0.0440719909965992, -0.032652005553245544, 0.054962433874607086, 0.07210838794708252, 0.009337031282484531, -0.04465440288186073, 0.004435798618942499, 0.06982897222042084, -0.0663822591304779, -0.06069428101181984, 0.03570260480046272, -0.07938439399003983, 0.06272023171186447, -0.09190215915441513, 0.04633525386452675, -0.028516439720988274, -0.027068568393588066, 0.04912940412759781, -0.044414132833480835, -0.0768280178308487, -0.07900112122297287, -0.02417663112282753, -0.011739318259060383, -0.07099264860153198, 0.032174769788980484, -0.025473827496170998, 0.026473620906472206, -0.026880161836743355, 0.019580431282520294, 0.017306260764598846, -0.02676851488649845, 0.06934557855129242, -0.04532581567764282, -0.08963444083929062, -0.027871953323483467, -0.0508863627910614, 0.019832657650113106, -0.018860701471567154, -0.0009805720765143633, -0.052601225674152374, 0.050868164747953415, -0.02369474247097969, -0.011502315290272236, -0.03531626984477043, 0.09633695334196091, 0.07824534177780151, 0.0561906099319458, 0.014502336271107197, -0.03995142877101898, -0.060294199734926224, 0.06139054521918297, 0.055742181837558746, -0.016836507245898247, 0.027213452383875847, -0.035508863627910614, -0.010528774932026863, 0.08579111844301224, -0.012474210932850838, -0.01701006107032299, -0.09255994856357574, -0.08538917452096939, -0.022668905556201935, 0.059963006526231766, -0.00490758428350091, -0.040022727102041245, 0.0030983558390289545, 0.04605671763420105, 0.007846667431294918, -0.01349619496613741, -0.005116232205182314, 0.06500262767076492, -0.03001839853823185, -0.04770580306649208, 0.04113361984491348, 0.0005114187952131033, 0.048489414155483246, 0.043324872851371765, -0.0251767598092556, -0.004708418156951666, 0.024154599756002426, -0.03377664089202881, -0.06048833206295967, 0.05086817219853401, 0.04488595575094223, 0.07740991562604904, 0.040619973093271255, -0.007867815904319286, 0.002873379737138748, -0.019708717241883278, -0.009714912623167038, -0.04637555405497551, 0.010311315767467022, 0.048368923366069794, 0.031524658203125, -0.028371069580316544, -0.08758645504713058, 0.07462819665670395, -0.03444315120577812, -0.036924686282873154, 0.007498626597225666, 0.026649588719010353, 0.03791184350848198, 0.04206085950136185, -0.04090925306081772, -0.026921674609184265, -0.045427050441503525, 0.09704124182462692, 0.07959360629320145, 0.020892472937703133, -0.03645096346735954, 0.06186593696475029, 0.0085389893501997, -0.0453948937356472, -0.08936899900436401, -0.06646715849637985, 0.061892248690128326, -0.07117367535829544, 0.018961679190397263, -0.020007433369755745, 0.07058579474687576, 0.03734678402543068, 0.04719929397106171, -0.03754989430308342, 0.05159095302224159, 0.04773658141493797, -0.04244522750377655, -0.031269341707229614, -0.09927976131439209, 0.0955289751291275, 0.04535979405045509, 0.03903769701719284, -0.01033876184374094, -0.0650133267045021, -0.02789345011115074, -0.06803697347640991, 0.08721385896205902, 0.05713943392038345, 0.011424502357840538, -0.05249224230647087, 0.0817071720957756, -0.06665050238370895, 0.027517246082425117, -0.10132350027561188, 0.00816434808075428, 0.03806673362851143, -0.06395480781793594, -0.002447764854878187, -0.01962300017476082, -0.018267957493662834, -0.08705353736877441, -0.06760882586240768, -0.06283197551965714, -0.056932684034109116, -0.03761640936136246, 0.07900559157133102, -0.03726979345083237, -0.02504904381930828, 0.06517083942890167, 0.026993345469236374, -0.010449543595314026, -0.025408871471881866, -0.08760908991098404, -0.005665539763867855, -0.08033019304275513, 0.03025173582136631, 0.0007468831026926637, 0.04346730187535286, 0.011832796968519688, -0.024997495114803314, -0.05992398411035538, -0.018477385863661766, 0.0869017094373703, -0.005649326369166374, 0.04674964025616646, -0.049496788531541824, -0.014608354307711124, 0.011684832163155079, 0.04621950909495354, 0.05826771259307861, -0.043602507561445236, 0.0032539735548198223, -0.056890349835157394, -0.022543661296367645, 0.03403692692518234, 0.032268915325403214, -0.01137411966919899, 0.06767498701810837, 0.054301489144563675, -0.02675677090883255, -0.0451657734811306, 0.07743111252784729, -0.03723759949207306, -0.044438984245061874, 0.06470329314470291, -0.007082867901772261, 0.06887415796518326, 0.0831131860613823, 0.04037130996584892, 0.057571034878492355, 0.010084027424454689, 0.02440938912332058, 0.06326033920049667, -0.05250724405050278, -0.049324940890073776, -0.09891942888498306, 0.010763774625957012, 0.019664809107780457, -0.034218061715364456, -0.026588134467601776, -0.08185587078332901, -0.08518091589212418, -0.006100582890212536, 0.028246311470866203, -0.019960757344961166, -0.015522879548370838, -0.05649539828300476, 0.016093499958515167, 0.017222411930561066, -0.03409727290272713, 0.027007371187210083, -0.030765589326620102, -0.08494484424591064, 0.09231539815664291, -0.05345839634537697, 0.08940298110246658, -0.023879224434494972, 0.012693356722593307, -0.041163310408592224, -0.0160730741918087, -0.09258437901735306, -0.011939223855733871, -0.03948526084423065, -0.0487247072160244, -0.060046371072530746, -0.03760829567909241, -0.011336597613990307, 0.072058767080307, -0.062157079577445984, 0.021524567157030106, -0.004999119788408279, -0.06884642690420151, 0.03648631274700165, 0.017440062016248703, 0.06763400882482529, -0.09356170892715454, 0.08805593103170395, -0.01990283839404583, 0.0995783731341362, -0.03234804794192314, 0.011676955036818981, 0.050973426550626755, -0.032270047813653946, 0.0725448727607727, 0.02249849960207939, 0.0011659003794193268, 0.06898611038923264, -0.037240926176309586, -0.01174040324985981, -0.08805191516876221, 0.061244893819093704, -0.0745251402258873, -0.07066753506660461, 0.1108299270272255, 0.048288680613040924, -0.0593305341899395, 0.10879874229431152, -0.018236329779028893, 0.028352733701467514, -0.04457886889576912, 0.058821216225624084, 0.0655956044793129, -0.07253614068031311, -0.07146837562322617, 0.0905945673584938, -0.021288488060235977, -0.006950641982257366, -0.016786547377705574, -0.083004429936409, 0.07431203871965408, -0.0034544968511909246, 0.06800971925258636, -0.04399700462818146, -0.07216979563236237, 0.023745190352201462, -0.06668656319379807, -0.018117275089025497, -0.08477674424648285, 0.08529563248157501, -0.04197826609015465, 0.03833073005080223, -0.01619233749806881, 0.048372361809015274, -0.04317861795425415, 0.05423740670084953, -0.0024710693396627903, 0.01893249712884426, 0.00536139402538538, -0.006779195740818977, 0.017249317839741707, 0.06874123960733414, 0.017044343054294586, 0.05291320011019707, 0.004071408417075872, 0.025648070499300957, -0.02496633678674698, 0.046405546367168427, 0.027322139590978622, 0.09076772630214691, 0.052294835448265076, -0.04490194097161293, -0.10391746461391449, 0.03263856843113899, -0.07037243992090225, -0.06267838925123215, 0.0006796331144869328, 0.034458331763744354, -0.06664548814296722, -0.00470142625272274, 0.017075326293706894, -0.0714859664440155, -0.06300187855958939, 0.09143473953008652, -0.08604039251804352, -0.06762076169252396, -0.028600549325346947, -0.027849335223436356, 0.0549168586730957, 0.07979485392570496, -0.033602241426706314, -0.0003934494743589312, -0.0029137595556676388, 0.07013014703989029, 0.02399357594549656, 0.043516721576452255, -0.03725402057170868, 0.05935794860124588, -0.03335482254624367, -0.05372295528650284, -0.01956699974834919, -0.05568689852952957, -0.04902706667780876, 0.07804100215435028, 0.03529999032616615, 0.06172158569097519, 0.01777300052344799, -0.007650548592209816, 0.030355239287018776, -0.030010264366865158, -0.04062797874212265, 0.03604493290185928, 0.09560099244117737, 0.03198770433664322, 0.034410346299409866, -0.020437031984329224, -0.07248349487781525, 0.044438038021326065, -0.07124635577201843, -0.05812820792198181, 0.0835173949599266, -0.02964245155453682, -0.05282307043671608, 0.031137114390730858, -0.02946881391108036, 0.0691625103354454, -0.05972332879900932, 0.07841785252094269, -0.0973055437207222, 0.08289788663387299, -0.06393953412771225, 0.0018271099543198943, 0.060312189161777496, 0.018109023571014404, -0.027121806517243385, -0.08134336024522781, 0.03754354640841484, -0.04199124872684479, -0.054779671132564545, 0.043229635804891586, -0.07977601885795593, -0.09094439446926117, -0.050821706652641296, 0.07561910152435303, -0.05770541727542877, 0.05237285792827606, -0.016803566366434097, 0.018156222999095917, 0.01662539504468441, 0.04418230056762695, -0.016094796359539032, -0.0003652545274235308, -0.03391312435269356, 0.0746188685297966, 0.06851685047149658, -0.08667957782745361, 0.03341653198003769, -0.06932438910007477, -0.05130892992019653, 0.010951518081128597, -0.0029647252522408962, 0.08058176934719086, 0.11604472994804382, 0.01462469156831503, -0.03930122032761574, 0.04249004274606705, -0.030310506001114845, 0.008250812068581581, 0.038753021508455276, -0.0869855061173439, 0.027483569458127022, -0.002028082497417927, 0.052382439374923706, 0.10792849212884903, 0.05074940249323845, 0.0974215716123581, 0.024919189512729645, 0.021429553627967834, 0.06557212024927139, -0.026744551956653595, -0.06547558307647705, 0.016967447474598885, 0.06950244307518005, 0.0669303759932518, 0.05882059037685394, -0.06974877417087555, 0.01433399599045515, 0.05887295678257942, 0.08683056384325027, 0.07237371057271957, -0.02932271920144558, 0.025538697838783264, -0.05791047215461731, -0.01927284523844719, -0.049303509294986725, -0.02938811108469963, -0.021537546068429947, 0.08444523811340332, -0.06767400354146957, -0.08662040531635284, 0.04403972625732422, 0.07962313294410706, 0.039159055799245834, -0.07094018161296844, 0.03943580389022827, -0.05953112244606018, -0.08706525713205338, 0.05140618979930878, -0.022000731900334358, -0.012413610704243183, 0.02233874797821045, -0.02908400446176529, 0.04609204828739166, -0.05630243569612503, 0.0098513662815094, 0.000556569080799818, 0.03340785577893257, -0.05684579163789749, 0.03890087828040123, 0.04932926967740059, 0.027669502422213554, 0.049898047000169754, -0.029711155220866203, -0.05338287726044655, 0.05180787667632103, 0.012522794306278229, -0.050477735698223114, 0.05891742929816246, 0.10291339457035065, -0.03447145223617554, -0.07961127907037735, -0.03545853868126869, 0.06036931276321411, 0.017394138500094414, 0.08544369786977768, 0.0870128870010376, -0.05973900482058525, -0.008596592582762241, -0.041379377245903015, -0.0010406860383227468, 0.010538061149418354, 0.07870570570230484, 0.04217847064137459, -0.00262100319378078, 0.03108619526028633, 0.021411098539829254, 0.021294577047228813, -0.05757648125290871, -0.026503169909119606, -0.029754450544714928, -0.008503029122948647, -0.011149048805236816, -0.06698720157146454, 0.009377173148095608, 0.0017249968368560076, 0.03813118860125542, 0.061822883784770966, 0.010197417810559273, 0.03541436791419983, 0.05806782469153404, 0.009134912863373756, -0.03455695882439613, -0.05723319202661514, -0.057777658104896545, -0.03875933960080147, 0.06124873086810112, -0.044500794261693954, 0.03149864450097084, 0.023750288411974907, -0.05418096110224724, 0.0389203242957592, 0.005745555739849806, 0.05852557346224785, 0.07643715292215347, 0.05767079442739487, -0.06173937767744064, 0.024650512263178825, -0.06457357853651047, 0.026513593271374702, -0.060145795345306396, 0.045965734869241714, -0.06640693545341492, 0.07308011502027512, -0.061732981353998184, 0.019284646958112717, 0.013798851519823074, -0.010616677813231945, 0.03900017589330673, -0.005342236254364252, 0.05186547338962555, 0.0740668922662735, 0.005680589936673641, -0.055895306169986725, 0.007662191987037659, -0.009742040187120438, -0.07208018004894257, 0.06899155676364899, -0.025083567947149277, -0.0018143816851079464, 0.06981691718101501, 0.0644441619515419, -0.0006026900373399258, 0.09137728810310364, -0.002074488438665867, -0.053101856261491776, -0.003289105836302042, -0.021739082410931587, 0.024820685386657715, -0.012587185017764568, 0.05912887677550316, 0.06450311839580536, 0.0317598357796669, -0.02234942838549614, -0.008018603548407555, -0.04386020451784134, 0.03312705457210541, 0.0863577127456665, -0.045452605932950974, -0.00680367648601532, 0.009410261176526546, -0.05728074163198471, -0.01094755157828331, 0.015089143067598343, 0.044513434171676636, 0.07597478479146957, 0.07665061205625534, -0.005228548776358366, -0.08090484142303467, -0.00844432134181261, -0.028380012139678, 0.02127620205283165, -0.09147202223539352, 0.05161032825708389, 0.04704373702406883, 0.02601003088057041, -0.06417348235845566, -0.02692495845258236, -0.05734832212328911, 0.073029063642025, 0.03956294804811478, -0.08904284238815308, -0.0649740919470787, -0.06015230342745781, 0.05617360770702362, -0.01682441681623459, -0.037566982209682465, -0.05043439939618111, 0.053690120577812195, 0.018443116918206215, 0.06876292824745178, -0.032157089561223984, 0.03968605771660805, -0.09207229316234589, 0.09288091212511063, -0.006955782882869244, 0.00961153395473957, 0.05271412804722786, 0.007747838273644447, 0.05947674438357353, 0.06713782250881195, -0.0784006416797638, 0.03526870161294937, -0.07063602656126022, -0.08399604260921478, 0.09087517112493515, -0.016899360343813896, -0.05105028301477432, -0.04260898754000664, 0.0396343357861042, 0.06868171691894531, 0.04711872711777687, 0.07772304117679596, -0.050821706652641296, -0.096474289894104, -0.01281009055674076, 0.04734107851982117, 0.08403749763965607, 0.032836709171533585, 0.004269398748874664, -0.08526090532541275, -0.006306528579443693, 0.056480906903743744, 0.05378599837422371, 0.07090086489915848, 0.03152595832943916, -0.03768135979771614, 0.06466607004404068, 0.047665487974882126, -0.015747398138046265, 0.06403548270463943, 0.0010500880889594555, -0.10233163088560104, 0.04991566762328148, 0.034858185797929764, 0.034695956856012344, -0.09879529476165771, 0.06940639019012451, -0.05167701840400696, -0.04734982177615166, -0.03738816827535629, -0.09505544602870941, 0.09835892915725708, -0.036482203751802444, -0.10089301317930222, 0.050501901656389236, -0.08674446493387222, 0.0055918279103934765, 0.041742656379938126, -0.07337675988674164, 0.0003577478928491473, 0.015717921778559685, -0.09323561936616898, 0.08155082911252975, -0.09277623146772385, 0.06985291838645935, 0.05185643956065178, -0.027702247723937035, 0.04040379077196121, -0.014905293472111225, 0.038359638303518295, -0.02719075046479702, 0.09731374680995941, -0.12243027240037918, -0.043704476207494736, -0.022612789645791054, -0.07081744819879532, 0.010882118716835976, -0.040263108909130096, -0.0907723680138588, -0.074065662920475, 0.005858588498085737, 0.061026494950056076, 0.05312975123524666, -0.08541525155305862, 0.07206140458583832, 0.07310385257005692, 0.03768555447459221, -0.05353544279932976, -0.03380550071597099, -0.013447239063680172, -0.03875720128417015, 0.010354239493608475, -0.019176337867975235, -0.0026944836135953665, -0.02217923477292061, 0.058918822556734085, 0.06929609924554825, -0.10252049565315247, 0.031127439811825752, -0.06405951082706451, -0.026932071894407272, -0.0010038414038717747, 0.07731281220912933, 0.04632857069373131, 0.06221836805343628, 0.0107883857563138, -0.0710035115480423, -0.06465812772512436, -0.03659462556242943, -0.0983857586979866, -0.10212356597185135, 0.0886339321732521, 0.005115253385156393, 0.029877986758947372, -0.08044497668743134, -0.053759701550006866, 0.026195615530014038, -0.05505302548408508, -0.06002863869071007, -0.002974723232910037, 0.04992751404643059, -0.020494654774665833, 0.04891017824411392, -0.02933545783162117, -0.01372557319700718, -0.023340562358498573, -0.01597997359931469, -0.0611879788339138, 0.002006049733608961, -0.09650260955095291, 0.003972184378653765, -0.0318172350525856, -0.07190337777137756, -0.026829088106751442, 0.08496753126382828, -0.0030691686552017927, -0.06014334410429001, -0.040481578558683395, -0.007169379387050867, -0.08165833353996277, 0.009180374443531036, -0.06592325866222382, 0.020363276824355125, -0.06708625704050064, 0.0076122279278934, 0.018480168655514717, -0.024725960567593575, -0.02907099574804306, -0.017991401255130768, 0.03079340234398842, 0.0033815838396549225, -0.0660974383354187, 0.06104609742760658, 0.07577087730169296, 0.026043402031064034, 0.012673581019043922, -0.035573750734329224, -0.053505320101976395, 0.07514679431915283, 0.028430726379156113, -0.022178996354341507, -0.042641013860702515, 0.08593360334634781, 0.026841027662158012, -0.0029307592194527388, 0.009904829785227776, -0.09736031293869019, -0.03407720476388931, 0.008217626251280308, -0.04764913022518158, -0.11048687994480133, 0.0053795152343809605, 0.07679648697376251, -0.06946776807308197, -0.06623325496912003, -0.09097952395677567, -0.020763248205184937, -0.0099548464640975, -0.12455932050943375, -0.06529077887535095, -0.0009838766418397427, 0.0035828249529004097, 0.00034745215089060366, 0.02868196740746498, 0.05597277358174324, -0.055313996970653534, 0.0007053293520584702, 0.009815783239901066, -0.06884447485208511, -0.0396229587495327, -0.010818935930728912, 0.0340961292386055, -0.037043508142232895, 0.021206844598054886, -0.02943391166627407, -0.014197385869920254, -0.012928065843880177, 0.0872139036655426, 0.013999433256685734, 0.030370457097887993, 0.037230994552373886, 0.00011796969920396805, -0.08881401270627975, 0.029113732278347015, 0.017352987080812454, -0.02665046602487564, -0.03251679614186287, -0.07780682295560837, -0.03075237013399601, 0.007437014486640692, -0.05669095739722252, 0.04298214986920357, -0.05500403791666031, -0.016284795477986336, -0.032368939369916916, 0.027629677206277847, -0.02621367760002613, -0.026313234120607376, 0.016629360616207123, -0.0045081088319420815, 0.05307694897055626, -0.07270895689725876, 0.06832954287528992, 0.09508544951677322, -0.009908531792461872, -0.06550277769565582, -0.000787037773989141, -0.0426357127726078, -0.059683121740818024, -0.03916894271969795, -0.00842461921274662, 0.00712899025529623, 0.05592835694551468, -0.07351729273796082, -0.09335322678089142, 0.08535873144865036, 0.0784778967499733, -0.11327502876520157, 0.012469317764043808, 0.04249949753284454, -0.04384274035692215, 0.050232842564582825, -0.0580424889922142, 0.011465352028608322, 0.03196961432695389, -0.0029602576978504658, -0.08628395944833755, -0.028185585513710976, -0.008275385946035385, -0.01911941170692444, 0.03450543433427811, 0.0650438442826271, -0.010098911821842194, -0.0344068706035614, -0.011223866604268551, 0.04071328043937683, -0.0931457057595253, 0.02030303329229355, 0.018880505114793777, -0.01132463663816452, -0.01087221410125494, 0.07017134130001068, -0.11075843125581741, 0.0537869930267334, -0.043851498514413834, 0.043534595519304276, 0.04435119032859802, 0.05800987035036087, 0.004940239246934652, -0.09079388529062271, 0.023297879844903946, -0.11786261945962906, 0.04297563061118126, 0.023432785645127296, 0.06586012244224548, -0.04944675788283348, 0.01241068635135889, 0.08497624844312668, -0.021435068920254707, -0.03719001263380051, -0.07090581953525543, 0.05248159542679787, 0.06921564787626266, -0.00353449210524559, -0.021794872358441353, 0.05039926990866661, 0.055860452353954315, 0.038881734013557434, 0.01219136267900467, 0.03662215173244476, 0.007885754108428955, -0.01774829439818859, 0.0402078814804554, -0.08571639657020569, 0.07087116688489914, 0.06919018924236298, 0.06804165244102478, -0.0267933439463377, -0.0616920180618763, 0.06225915998220444, -0.05170416831970215, -0.035079557448625565, 0.03827448561787605, 0.01012246310710907, 0.01669621653854847, -0.07357200980186462, 0.043144483119249344, -0.033185310661792755, 0.039519548416137695, -0.017227036878466606, -0.036403119564056396, -0.05653823912143707, 0.0479123555123806, -0.08181678503751755, 0.00768732326105237, 0.009713239967823029, -0.055178336799144745, -0.017175763845443726, 0.008244536817073822, -0.020477034151554108, 0.0134147759526968, -0.027410531416535378, 0.03857669234275818, -0.11941936612129211, 0.010931444354355335, -0.03243769705295563, -0.050475288182497025, 0.03995895013213158, -0.045344118028879166, 0.022810572758316994, 0.039351604878902435, 0.021620625630021095, -0.029124217107892036, 0.04177654907107353, 0.004861129447817802, 0.046297695487737656, 0.04603134095668793, 0.09209076315164566, -0.022434795275330544, 0.012325375340878963, -0.009416635148227215, 0.0830821767449379, -0.012727494351565838, -0.06852652877569199, -0.0020495615899562836, 0.10507047921419144, 0.08582814037799835, 0.05336355045437813, -0.03541335090994835, 0.08846411108970642, 0.07201962918043137, -0.006374620832502842, 0.04570832848548889, 0.08541844040155411, 0.012344134971499443, -0.0706489235162735, -0.03244580328464508, -0.02401023358106613, 0.07626582682132721, 0.0466335229575634, 0.05868229642510414, 0.06698764115571976, 0.05508168786764145, -0.04847307130694389, 0.017754916101694107, 0.10396414250135422, -0.08888242393732071, 0.01876671612262726, 0.055993955582380295, -0.060598842799663544, -0.03232257440686226, -0.08406809717416763, 0.03249550983309746, 0.042713578790426254, 0.04312020167708397, 0.048739779740571976, -0.04517168551683426, 0.05728317052125931, 0.044703565537929535, -0.06806629151105881, -0.09271641075611115, 0.08600591868162155, -0.06616140156984329, 0.06016272306442261, -0.01642310619354248, -0.06094489246606827, 0.06347107887268066, 0.06345688551664352, -0.051580674946308136, -0.08213268965482712, 0.07749199867248535, -0.0176031906157732, -0.05745002627372742, 0.07253847271203995, -0.05736628174781799, -0.10808227211236954, -0.04093718156218529, 0.0040860893204808235, 0.0931665301322937, 0.05290288105607033, 0.06090043857693672, -0.06708811223506927, -0.01090433169156313, -0.03775615617632866, -0.09283307939767838, 0.05192447826266289, 0.09292896836996078, 0.04337615147233009, 0.019655689597129822, 0.016304096207022667, 0.0038594300858676434, -0.02759886346757412, -0.08999506384134293, -0.009188344702124596, 0.02880879119038582, -0.06284258514642715, -0.017459679394960403, 0.056893378496170044, -0.009274512529373169, 0.059104353189468384, -0.06348247081041336, 0.05807684361934662, 0.017487332224845886, 0.08079742640256882, 0.09815634787082672, 0.060133837163448334, 0.06972440332174301, 0.061424124985933304, -0.09019476920366287, 0.08143651485443115, 0.05714546516537666, -0.034721098840236664, -0.02188296988606453, -0.05441880226135254, -0.011150245554745197, 0.08534637838602066, 0.026365337893366814, -0.07285579293966293, -0.018921054899692535, -0.0503866970539093, -0.059895437210798264, -0.05869147181510925, -0.03281073644757271, 0.03134922683238983, -0.07979928702116013, 0.07322779297828674, -0.08440864086151123, -0.002770373597741127, -0.08024805784225464, 0.04009716585278511, 0.0018133018165826797, -0.030374709516763687, 0.07709001004695892, -0.02769562602043152, 0.037225596606731415, -0.08096297085285187, 0.04186703637242317, -0.03449977934360504, -0.04604624956846237, -0.08743682503700256, -0.01918969117105007, 0.09025254100561142, -0.08943328261375427, 0.1003188043832779, -0.05873676761984825, 0.024089137092232704, -0.08769240230321884, -0.07557062059640884, -0.052938658744096756, 0.03402409702539444, -0.002323589287698269, 0.09020506590604782, 0.009239725768566132, 0.020236700773239136, 0.04484139010310173, -0.048195578157901764, -0.016532015055418015, -0.014025683514773846, -0.019905859604477882, 0.04411691427230835, 0.018085764721035957, -0.08489593118429184, -0.06162849813699722, 0.002660738304257393, 0.04205542430281639, 0.03244450315833092, -0.004144161473959684, -0.0601600706577301, -0.025550678372383118, 0.11172043532133102, -0.05195149779319763, -0.07207386195659637, 0.055362604558467865, 0.05576072260737419, -0.013402416370809078, -0.04893076419830322, 0.011773834004998207, 0.04935887083411217, -0.02731805481016636, -0.062479786574840546, 0.0368349589407444, 0.05154828727245331, 0.09623827785253525, -0.049345072358846664, -0.040100276470184326, -0.01778777688741684, 0.05972137302160263, -0.007299534045159817, 0.0027893180958926678, 0.06831393390893936, 0.06700079143047333, 0.04392309859395027, 0.061444446444511414, -0.07311086356639862, 0.07305948436260223, 0.030731961131095886, 0.03533755987882614, 0.02114233374595642, -0.08086267858743668, -0.022838018834590912, -0.060764867812395096, -0.0283513143658638, -0.0347224585711956, 0.07548419386148453, -0.042118728160858154, 0.021384380757808685, -0.07735498994588852, 0.06667065620422363, 0.023899953812360764, -0.055288560688495636, -0.037864793092012405, 0.031056975945830345, 0.025736669078469276, -0.022076571360230446, 0.07978029549121857, 0.022407783195376396, -0.021816739812493324, -0.06986358761787415, -0.041012976318597794, 0.10305734723806381, -0.06925072520971298, 0.05848588049411774, -0.07006842643022537, 0.04942929744720459, 0.04713897034525871, 0.0564306303858757, 0.05532735213637352, 0.05474194511771202, -0.0865098163485527, -0.06760787218809128, -0.05369230732321739, 0.03850007802248001, 0.03485773876309395, 0.011717048473656178, -0.06268512457609177, -0.004666599445044994, 0.042492520064115524, -0.06619520485401154, -0.08483631163835526, -0.017850039526820183, 0.09724690765142441, -0.06960272789001465, -0.015141526237130165, -0.04954928532242775, -0.11744372546672821, -0.07261830568313599, -0.0977923721075058, 0.05246357247233391, 0.054064635187387466, -0.09438455104827881, -0.03974458947777748, -0.08381679654121399, 0.04970964044332504, -0.025913283228874207, 0.026645787060260773, -0.001701738452538848, -0.07789451628923416, 0.05590807646512985, 0.015090216882526875, -0.019013281911611557, -0.014294778928160667, -0.027611682191491127, -0.06054503843188286, -0.07750359922647476, -0.05734831094741821, -0.011656827293336391, -0.022171277552843094, 0.07291436940431595, -0.0768323764204979, 0.043870959430933, 0.09697147458791733, -0.07322658598423004, 0.001064693438820541, -0.05080312490463257, -0.039291635155677795, 0.02409396320581436, -0.08759722858667374, -0.0833701491355896, 0.0016743856249377131, 0.051990654319524765, -0.09014301002025604, -0.0446743443608284, 0.0746358186006546, 0.05547815561294556, 0.025712240487337112, 0.0011969582410529256, 0.05216936767101288, -0.07195999473333359, 0.05376670882105827, 0.001302156364545226, 0.035097625106573105, -0.03092748299241066, 0.07350901514291763, -0.02977023273706436, 0.0005764091620221734, -0.04738595709204674, 0.023648736998438835, -0.09850195050239563, -0.0007239643018692732, -0.02041292004287243, 0.07078353315591812, 0.01561540924012661, 0.014373809099197388, -0.08171331882476807, -0.11286765336990356, 0.08709654957056046, -0.06454397737979889, 0.030096273869276047, 0.06845175474882126, 0.03079456277191639, -0.07553393393754959, -0.001984447706490755, 0.04109653830528259, -0.07647160440683365, -0.026780003681778908, -0.0034151142463088036, -0.03062310256063938, -0.042394913733005524, -0.10807622224092484, -0.007422529626637697, 0.06050023436546326, 0.011214176192879677, -0.04563750699162483, -0.04299340024590492, 0.056181441992521286, 0.05511019751429558, -0.011252920143306255, -0.10896772146224976, -0.04084919020533562, 0.021968044340610504, -0.02619495615363121, 0.0250546894967556, -0.03508657217025757, -0.06202283129096031, -0.050781138241291046, -0.06147615984082222, 0.09989824146032333, -0.12979359924793243, -0.04722059890627861, 0.05384204536676407, 0.0035121096298098564, -0.015452438965439796, -0.07801906764507294, -0.01995234750211239, -0.009068839251995087, 0.011313656345009804, 0.04002072289586067, 0.08574988692998886, -0.036500245332717896, 0.0881744846701622, 0.05460960045456886, -0.045257292687892914, 0.032949961721897125, 0.03744038939476013, 0.06964264810085297, -0.007531802635639906, -0.05374249815940857, -0.0009992027189582586, -0.0824987143278122, -0.046970706433057785, -0.06029126048088074, 0.049928806722164154, -0.05216202884912491, -0.01905173622071743, -0.01774485968053341, -0.04858001694083214, 0.018972542136907578, 0.022023916244506836, 0.009018105454742908, -0.09739331156015396, 0.04314536228775978, -0.043010517954826355, 0.06510890275239944, -0.10853758454322815, 0.017489394173026085, 0.024686232209205627, 0.04933185502886772, 0.019243625923991203, 0.034663744270801544, 0.017831064760684967, -0.0549614354968071, 0.07889357954263687, 0.008104187436401844, -0.03491799533367157, 0.05446869134902954, 0.02414075843989849, 0.007220858708024025, -0.05941804125905037, 0.0002331161085749045, 0.08233506232500076, 0.04887951537966728, 0.028596138581633568, -0.03777068853378296, 0.01875949837267399, -0.0873069018125534, 0.025253618136048317, -0.07977699488401413, 0.023105662316083908, -0.02481904625892639, 0.026599491015076637, 0.0006582357455044985, 0.028783230111002922, 0.06692822277545929, -0.007672477979212999, 0.03381234034895897, 0.062198005616664886, 0.007908242754638195, -0.0963137298822403, 0.033336058259010315, -0.06898632645606995, -0.002174406312406063, 0.08473777770996094, 0.030966866761446, -0.06485316157341003, -0.03830767795443535, 0.05298876389861107, -0.07396963238716125, -0.07618409395217896, -0.030205901712179184, -0.031579915434122086, -0.009357973001897335, 0.01977149024605751, 0.00862425472587347, -0.04380233213305473, -0.03634529933333397, -0.07206836342811584, 0.07680036872625351, 0.05959077179431915, -0.032651856541633606, 0.03478448837995529, 0.010629218071699142, -0.06502528488636017, -0.10240169614553452, -0.03326555714011192, 0.05040712282061577, -0.026658421382308006, -0.03827900439500809, 0.07752272486686707, -0.05990694463253021, 0.08557646721601486, -0.057472359389066696, -0.008077391423285007, -0.016379307955503464, 0.016879459843039513, 0.004778516478836536, 0.004188082180917263, -0.04814726114273071, 0.09073087573051453, 0.04493698105216026, 0.0683293417096138, 0.02796974778175354, -0.0743943378329277, -0.04597156122326851, -0.03051021695137024, -0.03834817185997963, -0.002792908111587167, -0.020829010754823685, -0.026801908388733864, 0.09787943959236145, 0.04070371761918068, 0.029551636427640915, -0.03461436554789543, -0.07234479486942291, 0.025616178289055824, 0.028758449479937553, 0.042433708906173706, -0.03858434036374092, -0.03508884087204933, -0.03667710721492767, 0.01085470337420702, 0.023419659584760666, -0.026777153834700584, 0.07162722945213318, 0.06057331711053848, 0.051885511726140976, 0.09080145508050919, -0.07216878235340118, -0.035646047443151474, 0.05390937998890877, -0.06573210656642914, 0.09563996642827988, 0.018908461555838585, 0.02968708798289299, -0.02802295982837677, -0.046341776847839355, 0.0880216658115387, -0.01397521048784256, -0.054809506982564926, 0.015664611011743546, 0.006093767937272787, -0.03351793810725212, -0.04378994554281235, -0.0852024108171463, -0.0153166763484478, 0.07544847577810287, -0.01439864095300436, 0.05686040595173836, -0.08503885567188263, 0.06828014552593231, -0.05525816231966019, 0.036816250532865524, 0.02515208162367344, 0.05282876268029213, -0.016301820054650307, -0.06376650184392929, -0.05421198904514313, 0.05032378062605858, -0.05812208727002144, 0.008971420116722584, 0.0699409767985344, 0.0651780217885971, 0.04187550023198128, -0.02342459373176098, -0.05335564166307449, -0.050363875925540924, 0.05035489425063133, -0.005900849122554064, -0.035652097314596176, 0.029912110418081284, -0.08795398473739624, 0.024388372898101807, -0.01093378383666277, 0.08321922272443771, -0.034571222960948944, 0.06348049640655518, 0.10147897899150848, 0.042714688926935196, 0.06746543943881989, -0.0611032210290432, -0.015865329653024673, 0.03892640024423599, -0.027903636917471886, -0.07896143943071365, -0.061860591173172, 0.009504735469818115, -0.046422652900218964, 0.08358602225780487, -0.0655345693230629, 0.05574815347790718, -0.04160911962389946, 0.027599476277828217, 0.011788513511419296, 0.10073859989643097, -0.02438206784427166, -0.10873349756002426, 0.08348514884710312, 0.07161454111337662, -0.07319607585668564, 0.044276975095272064, -0.10716912150382996, 0.01749636046588421, -0.008679823949933052, 0.08347564190626144, 0.009142397902905941, -0.0034896107390522957, 0.0209469273686409, -0.06070547550916672, 0.01020501833409071, 0.032052233815193176, 0.10748910903930664, 0.03064398095011711, 0.02874390222132206, -0.049175966531038284, 0.01457477267831564, 0.06374409794807434, -0.035778071731328964, 0.08437758684158325, -0.09302917867898941, 0.10189127922058105, 0.035168614238500595, 0.04094737023115158, -0.0008011795580387115, 0.019497353583574295, 0.0449935719370842, -0.001716947415843606, 0.03307449817657471, -0.027880389243364334, 0.011030036024749279, -0.08900269865989685, -0.028048310428857803, 0.024901803582906723, 0.02110612764954567, 0.0565456785261631, -0.06473357230424881, 0.01858503557741642, 0.021700691431760788, -0.008851980790495872, -0.08352726697921753, 0.04914394021034241, -0.02823728881776333, -0.0629630982875824, 0.040940817445516586, -0.07418649643659592, -0.08759775012731552, -0.014494904316961765, -0.05313994735479355, 0.03460577130317688, -0.1308394968509674, -0.045989323407411575, -0.02023562788963318, 0.011339006014168262, 0.03405272960662842, -0.01895601861178875, -0.03789002075791359, 0.026021936908364296, 0.044821690768003464, -0.02967052161693573, 0.029246870428323746, 0.02978747896850109, -0.006134349852800369, 0.04541531577706337, -0.014474254101514816, -0.05947856605052948, -0.06062735989689827, -0.05689794570207596, 0.012317529879510403, -0.015118086710572243, 0.0020577285904437304, 0.09703728556632996, -0.06621464341878891, 0.00921539030969143, -0.008796554058790207, 0.011677127331495285, 0.048505522310733795, 0.02063542790710926, -0.06677963584661484, 0.034274518489837646, 0.07262177765369415, -0.04613637179136276, 0.0076592471450567245, -0.034395910799503326, -0.012542499229311943, -0.0410885363817215, -0.020868822932243347, 0.06332182139158249, 0.04202907159924507, 0.0009193347068503499, 0.05623157322406769, 0.005896385759115219, -0.057174451649188995, 0.03894854709506035, 0.08383019268512726, 0.06290700286626816, 0.034338150173425674, 0.012294740416109562, 0.004035776015371084, -0.015014031901955605, 0.026064451783895493, -0.03186297416687012, -0.01080997847020626, -0.06376175582408905, -0.02326790802180767, -0.0070209507830441, 0.07808221876621246, 0.010248406790196896, 0.07628162950277328, -0.08428928256034851, -0.05450016260147095, -0.06986141204833984, -0.045972682535648346, 0.023883232846856117, 0.03382845222949982, 0.06390202790498734, 0.026330353692173958, 0.029373779892921448, -0.060694172978401184, -0.0027693987358361483, -0.06496884673833847, -0.07107329368591309, -0.011691143736243248, 0.021238142624497414, 0.09218327701091766, -0.03242632746696472, 0.046301376074552536, -0.07548794895410538, -0.059156615287065506, -0.00022884062491357327, -0.02698926068842411, -0.016833923757076263, -0.03500663489103317, 0.04219622537493706, -0.0073594991117715836, 0.040704596787691116, -0.06500061601400375, 0.06295496225357056, -0.009937078692018986, -0.040941547602415085, 0.0019388195360079408, -0.036643385887145996, 0.020779887214303017, -0.04678312689065933, -0.06916715204715729, 0.056393496692180634, 0.04879557713866234, 0.06740454584360123, 0.03345855697989464, 0.07128196209669113, 0.004046670161187649, -0.009001845493912697, 0.053903985768556595, 0.06351040303707123, -0.030636444687843323, -0.06171734631061554, 0.03267543390393257, -0.023430444300174713, 0.019799597561359406, -0.015050427056849003, -0.0754874050617218, 0.00032365944935008883, 0.10599338263273239, -0.057449374347925186, -0.01501934602856636, 0.018130633980035782, -0.025834793224930763, -0.022168705239892006, -0.047107331454753876, -0.02385418489575386, -0.021234944462776184, 0.07124416530132294, -0.01702803745865822, -0.022134415805339813, 0.05396156385540962, 0.015502768568694592, 0.11620363593101501, -0.008121144026517868, -0.02053835615515709, -0.0470573790371418, 0.025105565786361694, -0.011402243748307228, 0.025758441537618637, 0.011088035069406033, -0.04849134013056755, -0.021151309832930565, -0.06362715363502502, -0.044122420251369476, -0.02966725081205368, 0.031430281698703766, 0.0028474037535488605, -0.023150376975536346, 0.06438799202442169, -0.05067271739244461, -0.05959904193878174, 0.0280449315905571, 0.0881475955247879, 0.06684380024671555, -0.08086094260215759, -0.0003311045002192259, -0.06557555496692657, -0.02461531013250351, 0.03340700641274452, -0.003926305565983057, 0.05702685937285423, -0.0752476155757904, 0.027566948905587196, -0.08835460990667343, -0.03647923097014427, -0.044114790856838226, 0.08275307714939117, -0.07446685433387756, -0.07321703433990479, -0.10956525802612305, -0.06268466264009476, 0.013929523527622223, -0.08001010119915009, 0.0038083577528595924, -0.05148693546652794, -0.004271773621439934, -0.025837833061814308, -0.07322385907173157, 0.0946447104215622, -0.021000191569328308, 0.05265996232628822, -0.03174488618969917, 0.057566724717617035, 0.027299193665385246, 0.03468334674835205, -0.03368668258190155, 0.031177671626210213, -0.004118829034268856, 0.07568307965993881, -0.018179113045334816, -0.058464545756578445, -0.04366251081228256, 0.08278853446245193, 0.06290334463119507, -0.07710125297307968, -0.09042253345251083, 0.0011740295449271798, 0.07662201672792435, -0.00740061653777957, -0.09085670858621597, 0.06785184890031815, 0.045672379434108734, 0.10234812647104263, 0.0954514592885971, 0.016448305919766426, 0.000999380019493401, 0.07989364117383957, 0.09597565978765488, 0.010428490117192268, -0.01630023494362831, -0.011510448530316353, 0.08930753171443939, -0.02579941600561142, -0.059528909623622894, -0.06516752392053604, 0.03730145841836929, -0.016296178102493286, -0.07235133647918701, -0.07602881640195847, 0.05385846644639969, -0.07276654243469238, -0.029105540364980698, 0.00480086263269186, 0.028874505311250687, 0.014978719875216484, 0.0736197680234909, 0.026571044698357582, -0.06771538406610489, 0.005608479492366314, 0.0030798958614468575, -0.05679142102599144, 0.06651466339826584, -0.07799576222896576, -0.04953651502728462, 0.10134703665971756, -0.02178966999053955, -0.01614140160381794, -0.07281682640314102, 0.07316162437200546, 0.01992999017238617, -0.034493058919906616, 0.033955659717321396, -0.023150453343987465, -0.019106876105070114, -0.036730021238327026, -0.0008532792562618852, 0.08986274898052216, -0.07160351425409317, 0.09080447256565094, 0.06037188693881035, 0.044213224202394485, 0.02992008998990059, -0.019606327638030052, -0.03481215611100197, -0.07374072819948196, 0.07494024187326431, 0.02667471021413803, -0.013775457628071308, 0.05842893570661545, 0.050634268671274185, 0.05370509624481201, -0.01261175237596035, -0.02069486863911152, 0.04210435226559639, -0.0628722608089447, -0.07049845159053802, -0.06763756275177002, 0.06074078381061554, -0.11082406342029572, -0.06663241982460022, -0.08928504586219788, -0.016907447949051857, 0.045183632522821426, 0.07119693607091904, -0.06062403693795204, 0.06099165976047516, -0.07588580250740051, -0.06377852708101273, -0.06107061728835106, 0.02353735826909542, 0.007979446090757847, 0.04066278040409088, 0.08254583925008774, -0.11078327149152756, 0.048745978623628616, 0.06072310358285904, 0.03612183779478073, -0.07494956254959106, -0.053307272493839264, -0.038717906922101974, -0.011144882999360561, -0.040342237800359726, 0.02238478697836399, -0.06038753688335419, -0.07311388850212097, -0.01800590194761753, -0.029959173873066902, -0.05445707216858864, -0.0062864916399121284, 0.02952876314520836, -0.04559597000479698, -0.000667641288600862, 0.05188506469130516, -0.007614592090249062, 0.08790825307369232, -0.07279988378286362, 0.040703389793634415, -0.047650109976530075, -0.0500534251332283, -0.021954908967018127, 0.00910128653049469, -0.019655799493193626, -0.07442045211791992, -0.09390706568956375, -0.009011857211589813, -0.020085295662283897, -0.07392878830432892, -0.005148307420313358, -0.03340940549969673, -0.057498909533023834, -0.07585711777210236, 0.1038658618927002, 0.09487530589103699, 0.028701068833470345, 0.007630072068423033, -0.09010195732116699, -0.09106896072626114, 0.0471491701900959, 0.05963294953107834, 0.06004355102777481, -0.09090680629014969, 0.05309772491455078, -0.0015686338301748037, -0.12629641592502594, 0.038236670196056366, 0.06251690536737442, 0.10123396664857864, 0.10258188843727112, -0.04152711108326912, -0.017898397520184517, 0.1046844944357872, 0.08796525746583939, -0.085003562271595, -0.015145644545555115, -0.05817076563835144, 0.026219625025987625, -0.07109402120113373, -0.038048241287469864, 0.04270818084478378, 0.09477923065423965, -0.06137603148818016, -0.033919770270586014, -0.05328451097011566, -0.03734748438000679, 0.062258701771497726, 0.05329044908285141, 0.10373133420944214, 0.0197158120572567, -0.061691369861364365, -0.08317016065120697, 0.05755053833127022, -0.04737281799316406, 0.04931735247373581, -0.026237644255161285, -0.06653306633234024, 0.04442054405808449, -0.03590289130806923, 0.0014544606674462557, -0.03888066112995148, -0.09791643172502518, 0.010381351225078106, 0.028499076142907143, -0.05277464538812637, -0.0866481140255928, 0.06411164999008179, -0.07997369021177292, -0.06457199156284332, -0.060282763093709946, -0.04714765027165413, -0.04115134850144386, 0.0582788810133934, 0.021565549075603485, 0.0007242020219564438, 0.03570999205112457, -0.022089624777436256, -0.024206742644309998, -0.0020492447074502707, 0.03316470980644226, 0.025624144822359085, -0.002077523386105895, -0.003708201227709651, -0.08446425199508667, 0.008805835619568825, -0.06340105831623077, -0.04173879697918892, -0.02892991155385971, -0.042550042271614075, 0.018796399235725403, 0.03941600024700165, -0.06016397848725319, -0.004684044048190117, 0.01269278023391962, 0.04490454122424126, 0.02942216396331787, -0.034182820469141006, -0.044050589203834534, -0.028692154213786125, -0.005437396466732025, -0.00859360583126545, 0.04564574360847473, 0.05605868250131607, -0.035576872527599335, 7.180841930676252e-05, 0.02863805741071701, 0.05557771027088165, -0.057566288858652115, 0.036146532744169235, -0.08768907189369202, -0.002826751908287406, 0.0164907518774271, -0.09692685306072235, 0.04222531616687775, -0.04210285842418671, 0.06072685495018959, 0.028503285720944405, 0.056435126811265945, -0.036467842757701874, 0.02291346900165081, -0.0880754142999649, -0.08146116882562637, 0.03271210566163063, -0.013045595958828926, -0.0020583546720445156, 0.0099710738286376, 0.060472775250673294, -0.015645774081349373, -0.0802726075053215, -0.024516815319657326, -0.04424222931265831, 0.0500028021633625, 0.015421697869896889, -0.04822570085525513, 0.037704821676015854, -0.02080412581562996, 0.03551385551691055, 0.05945412442088127, -0.04023180156946182, -0.019527167081832886, 0.06311505287885666, 0.02030676230788231, 0.009703597985208035, -0.053170353174209595, 0.04565618932247162, 0.029902270063757896, -0.052045129239559174, -0.06122015789151192, 0.05609399080276489, 0.07880860567092896, -0.013723984360694885, 0.07456780970096588, 0.03270120918750763, -0.0434233620762825, -0.05137372389435768, 0.05249692499637604, -0.07240600883960724, 0.07834388315677643, -0.04042544588446617, -0.020831504836678505, -0.036567140370607376, -0.05724433809518814, -0.09944535791873932, -0.0474783331155777, -0.0641653910279274, 0.10549843311309814, -0.05861060693860054, -0.04330029338598251, -0.06640065461397171, 0.034837506711483, 0.04061560332775116, 0.02602817490696907, -0.07541900128126144, 0.0493682436645031, -0.027231521904468536, -0.03311280906200409, 0.013105962425470352, 0.07884485274553299, 0.0353679321706295, 0.06646013259887695, -0.02941424958407879, 0.005465135909616947, -0.06927445530891418, -0.08853258937597275, 0.022017359733581543, -0.07752618938684464, 0.044659391045570374, -0.03809052333235741, 0.048427335917949677, -0.1268814504146576, 0.03725492209196091, 0.08766830712556839, -0.012907852418720722, 0.028416644781827927, -0.00776412570849061, -0.10173669457435608, 0.09635947644710541, -0.04694376513361931, -0.03118160180747509, -0.08599985390901566, 0.039453886449337006, 0.005775082856416702, 0.033548031002283096, 0.09415210783481598, 0.067435123026371, -0.053186677396297455, 0.07403160631656647, -0.02397383563220501, 0.07998569309711456, 0.04823637008666992, -0.000561058521270752, 0.07701274752616882, -0.05321744084358215, -0.059783488512039185, 0.07306964695453644, -0.02451947331428528, -0.06655137985944748, -0.00528428889811039, 0.07430997490882874, 0.05684530362486839, 0.03210601583123207, -0.10390657931566238, 0.06997817009687424, -0.019866367802023888, -0.027708563953638077, 0.031866878271102905, 0.04078357666730881, 0.051917292177677155, -0.023318666964769363, -0.04182441160082817, -0.025228891521692276, -0.016441697254776955, -0.030572660267353058, -0.08112713694572449, -0.005268493667244911, 0.04039541259407997, -0.007703512441366911, 0.0024858249817043543, -0.025343043729662895, -0.08861104398965836, 0.052427347749471664, 0.03885042667388916, 0.0164719857275486, -0.02919374592602253, -0.07695090770721436, -0.002076312666758895, 0.040353648364543915, -0.04704982787370682, 0.05147015303373337, 0.08205773681402206, 0.048560284078121185, 0.010449369437992573, -0.07908953726291656, 0.07647499442100525, 0.06084742769598961, 0.055514223873615265, 0.0056481012143194675, 0.02215198241174221, -0.04698358103632927, 0.011736156418919563, -0.01316031999886036, -0.00175061309710145, 0.06554172188043594, 0.03790367767214775, 0.040541693568229675, 0.10684331506490707, 0.015506989322602749, -0.07583793252706528, 0.019625140354037285, -0.04123389720916748, 0.050260867923498154, -0.046424418687820435, -0.03853626176714897, 0.026264114305377007, 0.06977248936891556, 0.06021431088447571, -0.025697557255625725, 0.0216826144605875, -0.056070491671562195, -0.06622468680143356, -0.09860896319150925, 0.013656926341354847, -0.05673287436366081, 0.058985400944948196, 0.012148388661444187, -0.04066942259669304, 0.07742389291524887, -0.00036094040842726827, -0.07315857708454132, -0.028253184631466866, -0.023256853222846985, -0.030646784231066704, 0.04347126558423042, 0.03413348272442818, 0.003482575761154294, -0.05509970709681511, 0.019694004207849503, -0.08868340402841568, 0.038951385766267776, -0.049918461591005325, 0.008578545413911343, 0.0003418539126869291, -0.028086738660931587, -0.08511685580015182, 0.05742491036653519, -0.03249446675181389, -0.02952941693365574, -0.02878664992749691, -0.10218217968940735, -0.05603049322962761, -0.028740830719470978, 0.046533841639757156, 0.05151762440800667, 0.027808841317892075, 0.0653381273150444, -0.012308151461184025, -0.01815330795943737, -0.06637419015169144, -0.05528238043189049, -0.010294637642800808, 0.10810001194477081, 0.008441477082669735, 0.06685321778059006, -0.026995155960321426, 0.04688029736280441, -0.022221960127353668, -0.024145178496837616, 0.037977129220962524, 0.06082386523485184, 0.09944386780261993, -0.044764671474695206, -0.03857175633311272, 0.04410919174551964, -0.01660798490047455, -0.0181907769292593, -0.01767420955002308, -0.03220808133482933, 0.041517626494169235, -0.03687131032347679, 0.046273570507764816, -0.0737714022397995, 0.013576414436101913, 0.01632918231189251, -0.008235539309680462, -0.005297804716974497, 0.012097370810806751, 0.025473086163401604, -0.07036735117435455, 0.05305078253149986, 0.07620809227228165, -0.017350872978568077, 0.07160618156194687, -0.06261204183101654, -0.00178056419827044, -0.06477544456720352, -0.017861105501651764, -0.015865227207541466, 0.07714375853538513, 0.032768093049526215, -0.07821326702833176, -0.006453715730458498, -0.006099238526076078, 0.09260616451501846, -0.10674238950014114, 0.07498966157436371, -0.039795882999897, 0.011251936666667461, 0.05281227082014084, 0.034180574119091034, -0.0150085324421525, -0.015551035292446613, 0.019590279087424278, -0.060900669544935226, -0.01945846527814865, -0.030294831842184067, 0.08332286775112152, 0.05088823661208153, -0.06623422354459763, -0.06953439116477966, -0.07803719490766525, 0.10516273230314255, -0.011784978210926056, 0.004588100593537092, -0.08202113956212997, -0.018519194796681404, 0.018093854188919067, 0.03255222365260124, -0.008496884256601334, 0.055440496653318405, -0.025305738672614098, -0.0018666760297492146, -0.08640316873788834, -0.05491604283452034, -0.03027750924229622, -0.028917614370584488, -0.02615778148174286, -0.04321154206991196, -0.028119726106524467, -0.033396411687135696, -0.01801084727048874, -0.06471367925405502, -0.004496112931519747, -0.042223796248435974, -0.055073127150535583, 0.04639364033937454, 0.053812701255083084, 0.0461488701403141, 0.0007683111471123993, 0.0761406272649765, -0.036714281886816025, -0.04038877785205841, 0.015979787334799767, 0.05073858052492142, -0.025859959423542023, -0.06679023057222366, -0.06408269703388214, -0.05489996448159218, 0.0048705050721764565, 0.02274704910814762, 0.03799210116267204, 0.07965053617954254, -0.04695550352334976, 0.01684933342039585, -0.09635093063116074, -0.015226727351546288, 0.04442714899778366, -0.07144921272993088, 0.07235729694366455, 0.02968488819897175, -0.039519939571619034, 0.03799235820770264, -0.05385180562734604, -0.04559652879834175, -0.07296595722436905, -0.03716043382883072, -0.04757685959339142, 0.014404542744159698, -0.08864396810531616, -0.00683069322258234, 0.05174654722213745, -0.02771642990410328, 0.032031990587711334, -0.00013466182281263173, -0.04588190093636513, -0.01918003149330616, 0.07614244520664215, 0.0806116834282875, 0.020889734849333763, -0.024855049327015877, 0.008603786118328571, 0.03532077744603157, -0.026751836761832237, -0.0819176658987999, -0.04551147669553757, 0.09348311275243759, 0.017139431089162827, 0.026769382879137993, 0.04402093216776848, 0.00927048921585083, -0.014538339339196682, 0.059261683374643326, 0.06729410588741302, 0.010015016421675682, 0.06197028607130051, 0.005119184497743845, 0.09604566544294357, -0.06732377409934998, 0.0019373090472072363, -0.023113761097192764, -0.011924928054213524, -0.08271033316850662, -0.010610618628561497, 0.018671613186597824, 0.008307870477437973, -0.0027154521085321903, 0.07031026482582092, 0.0015042504528537393, -0.05159261077642441, -0.0433352068066597, 0.09712184220552444, -0.08753377944231033, 0.013750688172876835, -0.037180494517087936, -0.05904712527990341, 0.06968791782855988, -0.1053682193160057, 0.01958240382373333, -0.05279165506362915, 0.07382609695196152, -0.06145619601011276, 0.07009866833686829, -0.1115281879901886, -0.027779363095760345, -0.07126060128211975, 0.0720648318529129, 0.0454586036503315, 0.0707874670624733, 0.06460867822170258, 0.03980356827378273, 0.07875227928161621, -0.06042378395795822, -0.11278704553842545, -0.0758797824382782, 0.03402233123779297, 0.10016646981239319, -0.06680475920438766, 0.01599740795791149, 0.0187284704297781, -0.058620382100343704, -0.08208165317773819, -0.036054689437150955, -0.07802912592887878, -0.010821408592164516, -0.09731320291757584, 0.014940234832465649, 0.005312291439622641, -0.0029861072544008493, -0.02052917517721653, 0.013257895596325397, 0.012358472682535648, -0.060831934213638306, 0.0013286987086758018, -0.023016134276986122, -0.009887185879051685, -0.08926250040531158, 0.008246648125350475, -0.08888249099254608, 0.05546422302722931, 0.05570732429623604, 0.07939954102039337, -0.022500835359096527, -0.04519746080040932, -0.07773023098707199, 0.08766265958547592, -0.04499753937125206, -0.0495261512696743, -0.005600990727543831, 0.007504732348024845, -0.007008112967014313, -0.05742843449115753, -0.017592845484614372, -0.022817499935626984, 0.034415338188409805, -0.0285049919039011, -0.011965989135205746, 0.059096261858940125, 0.01732679270207882, -0.06597793102264404, -0.028124650940299034, 0.06718387454748154, 0.014900974929332733, -0.010994060896337032, -0.005282956641167402, 0.026794934645295143, 0.04510048031806946, 0.038624994456768036, 0.042235661298036575, -0.07762537151575089, 0.059924688190221786, 0.014114219695329666, -0.08627691119909286, -0.04486504942178726, -0.036615755409002304, -0.07374401390552521, -0.006327493581920862, 0.043615225702524185, 0.025513870641589165, -0.04390997812151909, 0.03756926581263542, 0.04777725413441658, -0.07020986080169678, 0.012011467479169369, 0.016158055514097214, 0.030431507155299187, -0.05068686977028847, -0.009288107976317406, -0.06903104484081268, -0.017799030989408493, -0.008056949824094772, -0.04319288954138756, 0.037864457815885544, 0.03858599066734314, 0.07894494384527206, 0.035513024777173996, 0.037435248494148254, -0.03471865877509117, 0.07629385590553284, 0.038485415279865265, 0.05518017336726189, -0.010607577860355377, 0.0480412058532238, -0.03311828151345253, -0.010871109552681446, -0.07430928945541382, -0.09838871657848358, 0.03578925505280495, 0.008332953788340092, 0.039284247905015945, 0.07946169376373291, -0.055778998881578445, -0.08747008442878723, 0.020834609866142273, -0.019916957244277, -0.08256334066390991, 0.10202707350254059, 0.02246182970702648, 0.058322954922914505, 0.05838445574045181, -0.04149816185235977, 0.016733812168240547, -0.04189513251185417, -0.08520787209272385, 0.012423583306372166, 0.04647918418049812, 0.06460303068161011, 0.05350450053811073, -0.0019428489031270146, 0.03021114133298397, 0.034956637769937515, -0.05616293475031853, -0.006019706837832928, 0.03694075345993042, -0.07962792366743088, -0.05292921140789986, -0.015387141145765781, 0.0722036361694336, -0.0012618841137737036, -0.08094913512468338, -0.0879872515797615, -0.07117484509944916, 0.030835092067718506, -0.10581653565168381, 0.07669351249933243, -0.060215093195438385, -0.08497192710638046, -0.05998200178146362, -0.0372682623565197, -0.047550469636917114, 0.033002231270074844, 0.022305544465780258, 0.051574550569057465, 0.061388008296489716, -0.07690972834825516, -0.05841876193881035, 0.08442327380180359, 0.010758798569440842, -0.049365293234586716, -0.06516318768262863, -0.02173456735908985, 0.04494544863700867, -0.015238122083246708, -0.0158836729824543, -0.045919932425022125, 0.07947240769863129, 0.05016598477959633, -0.0015890782233327627, 0.018912682309746742, -0.07107402384281158, 0.05069069564342499, -0.0011733334977179766, -0.040926750749349594, 0.021479427814483643, 0.07542493939399719, -0.006968884263187647, 0.01933642476797104, 0.001372131984680891, -0.04308188706636429, 0.021287309005856514, -0.06447724997997284, 0.03973033279180527, 0.035438600927591324, -0.010677661746740341, 0.08293414860963821, -0.05115404725074768, 0.006828003097325563, 0.020641159266233444, -0.04293680936098099, 0.02525303140282631, 0.06212750822305679, 3.937493602279574e-05, -0.014501256868243217, -0.09281418472528458, -0.06953594088554382, 0.008289556950330734, -0.01830524206161499, 0.05235955864191055, 0.07425426691770554, 0.0662965402007103, 0.00964172650128603, -0.010308203287422657, -0.04624251276254654, -0.020560890436172485, 0.03153605759143829, -0.04689889773726463, -0.03499877452850342, -0.06934627145528793, -0.05737905576825142, -0.0673595517873764, -0.08122860640287399, -0.0663774386048317, -0.04989377409219742, -0.05864013731479645, 0.045594170689582825, -0.028716128319501877, 0.046810612082481384, -0.05488547310233116, 0.08986689150333405, -0.044051311910152435, -0.05418472737073898, -0.05906520411372185, -0.007210083771497011, 0.044831790030002594, -0.08036640286445618, 0.0068413554690778255, 0.045058075338602066, -0.026632051914930344, -0.0370144285261631, 0.07859484851360321, -0.07680168747901917, 0.09053856879472733, 0.08609207719564438, -0.09085553139448166, 0.08479009568691254, -0.10295893251895905, -0.09257816523313522, 0.08196813613176346, 0.04528957977890968, 0.05365614965558052, -0.05018242821097374, 0.0011544538429006934, 0.06825000047683716, 0.03705345466732979, -0.06772246211767197, -0.050862912088632584, -0.0030383116099983454, 0.018512563779950142, 0.05641331896185875, -0.08895444869995117, -0.05695246905088425, -0.0835001990199089, -0.0039696055464446545, -0.09435170143842697, 0.005994090810418129, 0.003988491836935282, -0.0707898885011673, 0.05078722909092903, -0.08907194435596466, -0.12620946764945984, -0.000847021525260061, -0.05860297754406929, 0.026407599449157715, -0.05394366383552551, 0.02676168829202652, -0.0033799814991652966, -0.004703617189079523, 0.002468117279931903, -0.01944717951118946, 0.08672065287828445, 0.10355757921934128, 0.11254741251468658, 0.04929082468152046, 0.027797123417258263, -0.004921786952763796, 0.008389761671423912, 0.06953772157430649, -0.04110051691532135, 0.0035996343940496445, -0.07421135902404785, 0.037304963916540146, 0.043032433837652206, 0.07452557235956192, 0.014592143706977367, 0.05372212082147598, -0.04686403647065163, -0.01780235953629017, 0.015951091423630714, -0.08112181723117828, -0.03915926814079285, -0.02627670392394066, 0.06721536070108414, 0.054598208516836166, 0.10751979053020477, -0.08437521010637283, -0.0663803219795227, 0.028944317251443863, 0.028955578804016113, 0.005943553056567907, 0.05915458872914314, -0.0208037868142128, -0.0454675517976284, 0.02751629427075386, 0.03064722754061222, -0.025444742292165756, 0.001912510022521019, 0.06374675780534744, 0.035032764077186584, -0.061431530863046646, 0.02460743673145771, -0.09093808382749557, -0.08262098580598831, -0.07974380999803543, 0.022658172994852066, -0.044878654181957245, 0.05164339765906334, -0.04743935912847519, -0.04545879736542702, -0.0431763231754303, -0.04686765372753143, 0.0689743161201477, 0.004727824125438929, 0.028142279013991356, -0.06097070872783661, 0.026810768991708755, -0.10516015440225601, -0.0693725198507309, 0.05530841648578644, 0.09068001806735992, 0.054794151335954666, -0.007955742999911308, 0.006930817849934101, 0.06404299288988113, 0.04665564000606537, -0.06370266526937485, 0.02028195559978485, 0.0626199021935463, 0.024732770398259163, -0.04975385591387749, 0.03984423726797104, 0.05222654715180397, -0.010018802247941494, 0.06657323986291885, 0.023733191192150116, -0.05460752174258232, -0.030609922483563423, 0.029560094699263573, 0.0002973196969833225, 0.011040816083550453, -0.022800790145993233, 0.03253881260752678, -0.0013128709979355335, 0.008761465549468994, 0.04377839341759682, -0.11000719666481018, 0.030681828036904335, -0.010888774879276752, -0.07365632057189941, 0.03871830180287361, 0.01932358369231224, 0.034199055284261703, 0.0530584342777729, -0.07572107017040253, -0.015009768307209015, -0.06437109410762787, 0.003948059864342213, 0.02920559234917164, 0.029560817405581474, 0.04337742552161217, 0.08034501224756241, -0.028290262445807457, -0.04002639651298523, 0.057147540152072906, 0.0415770523250103, -0.032814815640449524, 0.0066478378139436245, 0.06465242803096771, -0.05349678918719292, -0.017070865258574486, 0.0434018149971962, -0.03676643222570419, 0.06746435165405273, 0.010593192651867867, 0.03256260231137276, 0.10494273900985718, 0.03647436201572418, -0.021388104185461998, 0.06314372271299362, 0.02392887882888317, -0.051926545798778534, 0.013175887055695057, -0.09655287116765976, 0.07021857053041458, 0.043709613382816315, -0.035627707839012146, 0.030732207000255585, 0.02832759916782379, 0.04357079416513443, 0.08861634880304337, 0.013504267670214176, 0.01703394390642643, -0.0512286014854908, 0.04588037356734276, 0.034003980457782745, -0.02872304618358612, 0.016398735344409943, 0.016663633286952972, 0.007088713813573122, 0.03645673394203186, 0.056734152138233185, 0.058403994888067245, 0.0021456321701407433, -0.029315726831555367, -0.12043081969022751, 0.04773781821131706, -0.08071554452180862, 0.06366607546806335, 0.026453953236341476, 0.05782584100961685, -0.06210780516266823, 0.0923558846116066, 0.0418417789041996, 0.037516579031944275, 0.027609195560216904, -0.08670157939195633, -0.06754504144191742, 0.014402981847524643, -0.033105622977018356, 0.0013446204829961061, -0.06809625774621964, -0.06629113852977753, -0.05773916095495224, 0.028743786737322807, 0.006342875771224499, -0.06218487769365311, 0.0890296921133995, -0.045082610100507736, 0.0013007279485464096, -0.08666247129440308, 0.050150636583566666, 0.05893077328801155, 0.015977732837200165, -0.0805373266339302, 0.06466590613126755, -0.024534834548830986, 0.022055309265851974, 0.046312619000673294, 0.04882623627781868, -0.03719620034098625, -0.05798545852303505, -0.011675514280796051, 0.04012131318449974, -0.015266865491867065, 0.03731430694460869, 0.005587685387581587, 0.06472919136285782, -0.07964761555194855, 0.01341722346842289, -0.09033548086881638, -0.038087133318185806, 0.013228651136159897, 0.03210146352648735, -0.043641600757837296, 0.0290403850376606, 0.05753866955637932, 0.0006597096798941493, 0.04387393221259117, -0.009112373925745487, 0.04120687395334244, -0.012184418737888336, -0.04858597740530968, 0.02189098484814167, -0.0415940061211586, 0.04807930067181587, 0.07471609115600586, 0.07522569596767426, -0.06741306185722351, -0.05173731595277786, -0.07027844339609146, -0.00532003678381443, 0.039798010140657425, 0.034825511276721954, -0.06804545223712921, 0.0717785581946373, 0.006458184216171503, -0.010556251741945744, -0.062465205788612366, 0.03811316192150116, -0.0031089147087186575, -0.017385849729180336, -0.0038945432752370834, 0.007730273995548487, -0.023037631064653397, -0.07383009046316147, 0.03208521008491516, -0.0608806237578392, 0.024831604212522507, 0.020773112773895264, 0.07959996163845062, -0.04474874958395958, 0.023680679500102997, -0.01080214511603117, 0.07531359791755676, -0.04993129149079323, -0.025468982756137848, -0.015913203358650208, 0.008380836807191372, 0.03103928454220295, 0.027282191440463066, -0.058842502534389496, -0.017101870849728584, -0.022085797041654587, 0.02606857940554619, -0.03889964893460274, -0.020996199920773506, -0.06835196167230606, 0.037307802587747574, -0.06425256282091141, -0.02417970634996891, 0.04448815435171127, -0.0117518724873662, 0.020824328064918518, 0.07318136841058731, -0.0582297220826149, -0.03557427227497101, 0.009151185862720013, -0.004580595064908266, -0.041064780205488205, 0.03774945065379143, -0.05209160968661308, 0.016371697187423706, 0.05176890268921852, 0.021095627918839455, -0.11153702437877655, -0.005258980672806501, -0.09108234196901321, 0.08398881554603577, 0.06849771738052368, -0.017526919022202492, 0.06275768578052521, 0.008056556805968285, -0.05994551628828049, -0.0852697342634201, 0.07740090042352676, 0.023553187027573586, 0.01305997371673584, 0.04521571099758148, -0.07483705878257751, 0.06286070495843887, -0.04262002184987068, -0.037164106965065, 0.058368291705846786, 0.04542669653892517, 0.03749847784638405, -0.07865295559167862, -0.04030501842498779, -0.07758481800556183, 0.022315723821520805, -0.057612344622612, -0.02847512625157833, 0.01570867747068405, 0.020597433671355247, 0.09507283568382263, 0.04656991362571716, 0.05637684836983681, 0.07101113349199295, -0.04844030365347862, 0.07912013679742813, 0.08525685220956802, -0.05173078924417496, -0.016616329550743103, 0.03605895861983299, -0.09366055577993393, 0.07179434597492218, -0.07606453448534012, 0.006346522364765406, -0.05029931291937828, 0.04834144189953804, -0.00036639394238591194, 0.057107798755168915, -0.03265662491321564, -0.083033986389637, -0.019115697592496872, -0.022877417504787445, -0.01040621567517519, 0.04244260489940643, -0.002532908460125327, -0.08597788959741592, -0.01897456869482994, -0.0658777579665184, -0.06543230265378952, 0.07793028652667999, -0.06966430693864822, -0.08691287040710449, 0.06237464025616646, -0.09731537848711014, 0.004216810222715139, 0.007662562653422356, 0.0737316682934761, -0.015371032990515232, -0.021472036838531494, 0.0716593861579895, -0.06960996240377426, -0.03763435035943985, 0.02188725583255291, -0.01742670312523842, -0.007856803014874458, -0.028493495658040047, -0.025036703795194626, 0.03254256397485733, 0.04755008593201637, -0.10192745923995972, -0.04414581134915352, 0.03891948238015175, -0.0740019902586937, 0.07535593211650848, 0.07264353334903717, -0.05919400975108147, 0.08617251366376877, -0.05594414472579956, -0.06653282046318054, -0.05969861149787903, 0.025984464213252068, 0.009404224343597889, -0.02383769489824772, 0.00859114620834589, -0.040196407586336136, 0.038626737892627716, 0.06999123096466064, -0.06918098032474518, -0.07084829360246658, 0.0648840144276619, -0.06617091596126556, -0.08759657293558121, 0.026621155440807343, 0.06537801027297974, 0.01751403696835041, -0.051609162241220474, 0.025402989238500595, -0.02033771574497223, 0.0030473684892058372, -0.07956842333078384, -0.10119684040546417, -0.03599601611495018, 0.043751414865255356, 0.049626778811216354, 0.052459605038166046, -0.03405611217021942, -0.0399588979780674, 0.030534714460372925, -0.04783245548605919, 0.06635084748268127, 0.05040140077471733, -0.014425029046833515, 0.004911546129733324, -0.025480348616838455, 0.019768288359045982, 0.042921263724565506, 0.03630255162715912, -0.008673596195876598, 0.027529433369636536, -0.048585500568151474, -0.09306169301271439, 0.031255703419446945, -0.06217852234840393, 0.02431187406182289, 0.018880071118474007, -0.014443604275584221, -0.014532934874296188, 0.017736759036779404, -0.059633418917655945, -0.08149304240942001, -0.03988678380846977, 0.06025899946689606, 0.06524281203746796, 0.055359914898872375, -0.04542287066578865, -0.03313750773668289, 0.043081656098365784, 0.0037047453224658966, 0.057540666311979294, 0.008506221696734428, 0.0020478281658142805, 0.034326959401369095, -0.029760966077446938, 0.02172788791358471, -0.02773444727063179, -0.0507044717669487, -0.022536706179380417, -0.0935489609837532, 0.04091963917016983, -0.02696055918931961, -0.009172136895358562, 0.0027095857076346874, -0.029276400804519653, 0.04205798730254173, 0.021354984492063522, 0.032064229249954224, -0.07631119340658188, 0.05878942459821701, 0.09390480816364288, 0.030424004420638084, -0.08570948243141174, 0.01176499854773283, 0.0342550165951252, 0.0640660747885704, -0.05514756962656975, -0.04136127978563309, 0.08798243850469589, -0.058516159653663635, -0.02782951481640339, 0.044060349464416504, -0.07116324454545975, 0.07063130289316177, 0.007081234827637672, -2.8147624107077718e-05, 0.07873562723398209, -0.027862535789608955, -0.07456889003515244, 0.024536028504371643, -0.022658662870526314, -0.056274138391017914, 0.05358850955963135, -0.01043262891471386, -0.005493044853210449, -0.018155165016651154, 0.030606728047132492, 0.018887178972363472, -0.00926992204040289, -0.031914182007312775, 0.051410652697086334, 0.07036920636892319, 0.05112624540925026, 0.004429088439792395, 0.07706288248300552, 0.04637362062931061, 0.04717915877699852, 0.10620007663965225, 0.0013552340678870678, -0.04016359895467758, -0.08935566246509552, 0.013843928463757038, 0.026078809052705765, -0.06491753458976746, 0.025721145793795586, 0.012284340336918831, -0.010122527368366718, -0.014575589448213577, 0.07551498711109161, -0.04908989369869232, 0.10201690346002579, 0.019723957404494286, -0.030826257541775703, 0.0005476460210047662, -0.08456499874591827, 0.09158468246459961, 0.08343559503555298, 0.039127152413129807, -0.024284111335873604, -0.06997893750667572, 0.10453135520219803, 0.04236459732055664, -0.008816762827336788, -0.01688830927014351, -0.11749330163002014, 0.06174694001674652, 0.05706307664513588, -0.026121055707335472, 0.018945129588246346, -0.06004462391138077, -0.048945583403110504, 0.02270480804145336, 0.015290704555809498, 0.0705455020070076, -0.016835980117321014, -0.05346735939383507, -0.06360147148370743, 0.017038077116012573, 0.05755404755473137, -0.039919886738061905, 0.01280177105218172, 0.050434257835149765, -0.04430312663316727, 0.08478256314992905, 0.00794309563934803, 0.007788104470819235, -0.07010295242071152, 0.019605956971645355, 0.07130633294582367, -0.08265258371829987, 0.04631781578063965, 0.0564894825220108, -0.06840747594833374, 0.06464273482561111, -0.09500845521688461, 0.04176628589630127, 0.08801502734422684, 0.018752356991171837, -0.06272675096988678, -0.1075790673494339, 0.014427362941205502, 0.12290278822183609, 0.027668198570609093, -0.02363637462258339, 0.08028976619243622, 0.08415614813566208, 0.07102157920598984, -0.04869365319609642, 0.00568806380033493, -0.012236096896231174, -0.05040489509701729, -0.01460434403270483, -0.05918526649475098, -0.03730195760726929, -0.03564101830124855, 0.0034655719064176083, 0.004951578564941883, 0.04816514626145363, -0.0026516267098486423, 0.04121232405304909, 0.03625201806426048, 0.058719076216220856, 0.05868728831410408, -0.018161291256546974, 0.040198687463998795, 0.07756750285625458, 0.005749698728322983, 0.08274053037166595, 0.07159679383039474, -0.07254362851381302, -0.03296240419149399, 0.104117251932621, 0.030561119318008423, 0.02309453673660755, 0.006976848468184471, 0.05990353226661682, 0.03416779264807701, -0.023913970217108727, -0.0010334508260712028, 0.061039697378873825, 0.029790719971060753, 0.04070667549967766, 0.031013373285531998, -0.06502453982830048, -0.02520853839814663, 0.033541589975357056, -0.07933136820793152, -0.02298698201775551, 0.035714440047740936, 0.0903700590133667, 0.02711118757724762, 0.028054265305399895, 0.03154481574892998, 0.02966272458434105, 0.07707575708627701, 0.09763802587985992, -0.03618185967206955, 0.07731270045042038, -0.04301098734140396, -0.013320968486368656, 0.06478942930698395, -0.0491718091070652, -0.02511139214038849, -0.039877377450466156, -0.0600353367626667, 0.06597013771533966, 0.07484882324934006, 0.05769805610179901, -0.02685270830988884, 0.029997844249010086, -0.028844039887189865, 0.04175308719277382, -0.007191305048763752, -0.0239311084151268, 0.029930638149380684, 0.047148317098617554, 0.012991064228117466, -0.033917948603630066, 0.05171981826424599, -0.0316506028175354, -0.05395787954330444, 0.022514108568429947, -0.006761184893548489, -0.05903617665171623, -0.03617081418633461, -0.021613990887999535, -0.04573579505085945, -0.058201730251312256, 0.01857748068869114, -0.08171961456537247, 0.04573873430490494, 0.02017049305140972, 0.09394398331642151, 0.06902074068784714, -0.07275696098804474, -0.0437471978366375, -0.05462375655770302, -0.056351687759160995, 0.06000075116753578, 0.03856818005442619, -0.06306632608175278, 0.04338478669524193, -0.04543250426650047, 0.050420019775629044, 0.04030323773622513, 0.05215708911418915, 0.04349963366985321, 0.056064117699861526, -0.049467533826828, 0.016088254749774933, 0.0012525009224191308, -0.058118786662817, -0.04410514608025551, 0.06478352844715118, -0.0048101358115673065, 0.053495142608881, -0.044693559408187866, -0.07260628044605255, -0.08079271763563156, -0.035015787929296494, 0.03961804881691933, 0.05061103776097298, -0.0411866158246994, -0.06805717945098877, 0.030367078259587288, 0.03324512392282486, 0.051453519612550735, -0.0030511273071169853, -0.01680581271648407, 0.01633567549288273, -0.020129607990384102, 0.08269225060939789, 0.03639676794409752, -0.0520341582596302, -0.08143159747123718, -0.03775082528591156, 0.06531180441379547, -0.08693742007017136, 0.007149226497858763, -0.03266584500670433, -0.0678686872124672, -0.07907046377658844, -0.08040595054626465, -0.04824374243617058, -0.025201601907610893, 0.05620972439646721, -0.03929121419787407, -0.04849866032600403, 0.02353478968143463, 0.03467864915728569, -0.0421830490231514, -0.061793871223926544, 0.045403528958559036, 0.04289361834526062, 0.01649049110710621, -0.08166651427745819, 0.07078855484724045, 0.029611967504024506, 0.013380618765950203, 0.06099279597401619, 0.0074671171605587006, 0.07051075994968414, -0.006589211523532867, 0.05959818512201309, 0.07974868267774582, 0.037273164838552475, -0.025257496163249016, -0.02952681854367256, -0.06088503450155258, -0.05081840604543686, 0.07709506154060364, -0.08351144939661026, 0.07278529554605484, -0.08273544907569885, -0.0028867237269878387, 0.06780748069286346, 0.058539312332868576, -0.003236452816054225, -0.023508058860898018, 0.07713789492845535, 0.10342199355363846, -0.003660482820123434, 0.043134354054927826, -0.03773418813943863, -0.06398241966962814, -0.04363728314638138, -0.013708008453249931, 0.048626918345689774, -0.06230968236923218, 0.0011095911031588912, -0.025270827114582062, 0.034033503383398056, 0.07177292555570602, -0.018306992948055267, 0.047673843801021576, -0.04938501492142677, 0.013206375762820244, 0.09638026356697083, 0.040965452790260315, 0.07812704145908356, 0.0037043194752186537, -0.007111558690667152, 0.027006790041923523, 0.040201954543590546, -0.037775833159685135, 0.0027532472740858793, -0.09942788630723953, 0.047564227133989334, -0.05873841792345047, 0.010417615063488483, 0.04646869748830795, 0.03701530396938324, 0.039651744067668915, -0.06408517807722092, 0.015434468165040016, -0.0008902775589376688, 0.05214758217334747, 0.04070538282394409, -0.006606894079595804, -0.011963887140154839, 0.057889554649591446, -0.0045026629231870174, 0.02130417339503765, -0.08473382145166397, 0.07684853672981262, -0.04126790910959244, 0.04744652658700943, 0.06042250990867615, -0.03263767063617706, -0.03234618529677391, 0.06737295538187027, -0.005121428519487381, -0.07900448143482208, 0.026725605130195618, -0.08327668905258179, -0.05193673446774483, 0.006715873721987009, 0.04984941706061363, 0.007243792526423931, -0.005254287272691727, 0.06311678141355515, 0.04387040063738823, 0.05178073048591614, -0.06347446888685226, 0.07949532568454742, 0.06007402390241623, -0.08461152017116547, 0.03913642466068268, 0.0873451679944992, 0.05452292785048485, -0.022582819685339928, -0.047854382544755936, -0.015412633307278156, -0.02934829704463482, -0.037312764674425125, -0.08346031606197357, 0.033412832766771317, -0.03383800387382507, -0.06796327978372574, -0.029876261949539185, 0.06911662966012955, -0.08082972466945648, 0.04109179228544235, -0.08188182860612869, 0.017846794798970222, -0.008366684429347515, 0.018316857516765594, -0.016447529196739197, 0.01615101285278797, -0.01158810593187809, 0.023782266303896904, 0.0287824347615242, 0.006144469603896141, 0.01243402436375618, -0.017396317794919014, -0.014344952069222927, 0.05130681395530701, -0.005790619645267725, -0.020594023168087006, -0.07192928344011307, -0.08259984850883484, -0.02230623923242092, 0.011359612457454205, 0.05741706117987633, -0.037441711872816086, 0.012289047241210938, -0.007911110296845436, 0.0623973049223423, 0.06407502293586731, -0.057630233466625214, 0.017706453800201416, -0.03538865968585014, -0.07772567868232727, 0.08078460395336151, -0.06576882302761078, 0.02127126418054104, -0.11226721107959747, -0.03918730095028877, -0.031585611402988434, 0.029837070032954216, 0.04950661212205887, -0.023921729996800423, 0.035761620849370956, 0.05758783966302872, 0.03200378268957138, 0.04249517247080803, 0.022079283371567726, -0.030120933428406715, 0.05562836304306984, -0.01750028133392334, -0.08137664943933487, 0.014052187092602253, -0.06298030912876129, -0.03865693882107735, -0.01157065387815237, -0.04972017928957939, 0.044385217130184174, -0.044749800115823746, -0.020043373107910156, -0.05255340784788132, 0.004364164546132088, -0.0849112868309021, -0.08873339742422104, 0.030861511826515198, 0.0612829327583313, -0.09182360023260117, -0.04621626436710358, 0.06856557726860046, 0.06147162243723869, 0.03649924322962761, 0.02659953013062477, -0.07524801045656204, -0.09794376790523529, -0.08582984656095505, -0.07147682458162308, -0.07679104059934616, 0.04283712059259415, -0.02729702554643154, -0.07394113391637802, -0.023494260385632515, -0.03001086786389351, -0.030033059418201447, 0.027898909524083138, -0.09186777472496033, 0.0025716819800436497, 0.05921662971377373, -0.05976300314068794, 0.07910003513097763, 0.01722370646893978, 0.03901999816298485, 0.02653992734849453, -0.06698905676603317, 0.03939615562558174, -0.11170203238725662, 0.07685993611812592, -0.06202414631843567, -0.033251356333494186, 0.09350705146789551, 0.057288482785224915, 0.018129311501979828, -0.07854092866182327, 0.03251994028687477, -0.09465394914150238, 0.05846225097775459, -0.022047195583581924, 0.03510047122836113, -0.08716096729040146, -0.006507985293865204, -0.023437462747097015, -0.04227256774902344, -0.0032753273844718933, -0.06777970492839813, 0.08486364781856537, -0.01710394211113453, 0.09429574757814407, -0.023775242269039154, 0.06331092864274979, -0.05608874931931496, 0.029713526368141174, -0.004163577686995268, 0.06690070033073425, 0.04311491921544075, 0.08788648247718811, 0.054597511887550354, 0.05512817203998566, -0.051284678280353546, -0.005792639218270779, -0.01980477198958397, -0.07333434373140335, 0.005552390590310097, -0.05035741627216339, 0.060313113033771515, -0.006825741846114397, -0.052161265164613724, -0.060748372226953506, 0.10026141256093979, 0.014751195907592773, 0.06794410198926926, -0.060921456664800644, -0.07175354659557343, -0.05645258352160454, 0.05149870365858078, -0.04588329792022705, -0.054975271224975586, 0.020237209275364876, 0.08231011778116226, 0.05980624258518219, -0.08037373423576355, -0.012257303111255169, 0.001253733062185347, -0.1003490760922432, -0.07126010954380035, -0.034529536962509155, -0.08588742464780807, -0.06212171912193298, -0.04958070069551468, 0.008979971520602703, 0.016114821657538414, -0.036870479583740234, -0.052609823644161224, -0.01509871892631054, -0.08119771629571915, -0.007267987355589867, 0.004349541384726763, -0.014237731695175171, -0.0015772348269820213, -0.020363055169582367, 0.04155752807855606, -0.1028754860162735, 0.009933830238878727, -0.01749211922287941, 0.06311716139316559, -0.04536149650812149, -0.09430115669965744, -0.03320665284991264, -0.0723281055688858, 0.08825617283582687, -0.01691194251179695, 0.08232518285512924, -0.061503298580646515, -0.03328946232795715, -0.07035341113805771, 0.058370549231767654, -0.0609058178961277, 0.03583597019314766, 0.011469660326838493, 0.07398182153701782, 0.00791873224079609, 0.011192151345312595, -0.05508854240179062, -0.0505366176366806, 0.048428796231746674, -0.034174200147390366, -0.0739108994603157, 0.07913172245025635, 0.05690665915608406, -0.03970084339380264, -0.017469774931669235, 0.061213381588459015, 0.018853873014450073, -0.040970802307128906, -0.06860031187534332, 0.027142638340592384, 0.03321355953812599, 0.0067855315282940865, -0.07570936530828476, -0.030950184911489487, 0.03104718215763569, 0.04353804141283035, 0.02931053750216961, 0.055138807743787766, 0.0310685895383358, -0.020503275096416473, 0.00853021815419197, 0.05685526132583618, -0.08382435142993927, -0.0269927941262722, 0.07228311896324158, 0.06335163861513138, 0.01572233997285366, 0.0012717293575406075, 0.06302321702241898, 0.1024649441242218, -0.059060752391815186, -0.09419550001621246, 0.07419060915708542, 0.06757576018571854, 0.07167224586009979, -0.07482121139764786, -0.040721483528614044, -0.031084252521395683, -0.010653638280928135, 0.06292158365249634, -0.06747083365917206, -0.011160085909068584, -0.0797591507434845, 0.05805158615112305, -0.06658113747835159, 0.002166248159483075, 0.05022043362259865, -0.034500617533922195, -0.01119265053421259, 0.06437832862138748, -0.03414871171116829, 0.027004646137356758, 0.026101941242814064, -0.031004656106233597, 0.05762970447540283, -0.06873603910207748, -0.07238974422216415, -0.07762552052736282, 0.04710664972662926, -0.0874890610575676, -0.06650673598051071, -0.029884422197937965, 0.07801279425621033, 0.07494425773620605, -0.02940048649907112, -0.006719952914863825, -0.033551350235939026, 0.026746587827801704, -0.08119730651378632, -0.05469971522688866, -0.01583077944815159, 0.010277179069817066, 0.051470618695020676, -0.08928892761468887, -0.05298144742846489, -0.07155637443065643, -0.050079237669706345, -0.02986760064959526, -0.010819535702466965, -0.052037786692380905, -0.020435703918337822, 0.07603629678487778, 0.05544475466012955, -0.013176240026950836, -0.06083380803465843, 0.07544011622667313, -0.0716705247759819, 0.07301050424575806, -0.05294803902506828, 0.0031492554116994143, -0.01593334786593914, -0.05960344523191452, -0.0599469393491745, -0.04701890051364899, 0.05868896096944809, -0.03249365836381912, -0.060071587562561035, -0.05038103461265564, 0.031389810144901276, -0.055802199989557266, -0.019038202241063118, -0.09998298436403275, -0.04569109529256821, -0.027641290798783302, -0.03773927688598633, 0.06217129901051521, 0.05306188389658928, 0.04979900270700455, 0.06038399040699005, -0.02574782818555832, -0.05393539369106293, 0.02681245282292366, 0.05720186606049538, 0.028306078165769577, -0.008730685338377953, 0.022351466119289398, 0.004917898215353489, 0.06546451151371002, -0.024585284292697906, -0.055332526564598083, 0.03898997977375984, 0.004215758293867111, -0.07773835211992264, -0.06533975154161453, 0.04726089537143707, 0.0007989304140210152, -0.0030262679792940617, -0.06423552334308624, 0.056248802691698074, -0.02007191814482212, -0.01587655395269394, 0.07708766311407089, 0.08848622441291809, 0.059712015092372894, 0.03260791301727295, 0.003910132683813572, -0.025579016655683517, -0.08573020249605179, 0.056110985577106476, 0.03401687368750572, 0.025701899081468582, -0.01600036583840847, 0.07398671656847, -0.038170263171195984, 0.07410632818937302, -0.026429321616888046, -0.03427213057875633, -0.04953258857131004, -0.04997868835926056, -0.0036352500319480896, -0.04857216775417328, 0.08202843368053436, -0.08482808619737625, 0.07201560586690903, 0.0033430398907512426, -0.015009340830147266, -0.02458478882908821, -0.009374205023050308, -0.10425636917352676, 0.1283705085515976, -0.0013520580250769854, -0.031664129346609116, -0.07109102606773376, -0.05688214302062988, -0.05765474960207939, 0.01638570986688137, -0.09317972511053085, 0.07326725870370865, -0.029458466917276382, 0.03590621054172516, 0.04331117123365402, 0.034724704921245575, -0.08980657905340195, -0.029362650588154793, 0.056763987988233566, 0.02155778929591179, -0.02426859922707081, 0.027077477425336838, 0.03057781793177128, 0.09372357279062271, -0.021935412660241127, 0.079280324280262, -0.0427289754152298, -0.034802597016096115, -0.10204494744539261, -0.05122428759932518, -0.050790369510650635, -0.046594735234975815, 0.02328280359506607, 0.08884872496128082, 0.011134905740618706, -0.017679212614893913, 0.06799094378948212, -0.01940559782087803, -0.030960550531744957, -0.05326506122946739, 0.015886643901467323, 0.028218165040016174, -0.07214657962322235, 0.03589986264705658, 0.08138008415699005, -0.0036892290227115154, 0.04180547222495079, -0.02693674899637699, -0.04798367619514465, 0.0053009451366961, -0.01179844606667757, -0.0018498031422495842, 0.04175376147031784, 0.021051689982414246, 0.026038365438580513, -0.03622616082429886, -0.04420888051390648, 0.07975628226995468, -0.05578827112913132, -0.05378981679677963, 0.04430466145277023, -0.036949221044778824, -0.031012406572699547, -0.06623483449220657, 0.09316110610961914, -0.08488566428422928, -0.024428848177194595, 0.04002773016691208, 0.004602887202054262, 0.09434907883405685, -0.050420623272657394, 0.08388359844684601, -0.04405903071165085, 0.014455189928412437, -0.002912587486207485, 0.009329459629952908, 0.10327509045600891, -0.043413981795310974, -0.05867232382297516, -0.06767454743385315, 0.04920341819524765, 0.03949539735913277, 0.0033598090521991253, 0.06197835132479668, -0.04055153951048851, 0.06554870307445526, 0.07111380249261856, -0.004614312667399645, 0.026438912376761436, -0.035717811435461044, -0.04230310395359993, 0.024359706789255142, -0.037076324224472046, 0.044239163398742676, 0.08030758798122406, -0.036189209669828415, 0.06784260272979736, 0.017540622502565384, -0.06618215888738632, -0.10248196870088577, -0.0026248202193528414, -0.07030856609344482, -0.036797720938920975, -0.07203850150108337, -0.004119439981877804, -0.019165681675076485, -0.071589395403862, -0.07526268064975739, -0.07103095203638077, -0.09657279402017593, 0.007009919732809067, 0.02546701207756996, -0.0001592230109963566, 0.018600180745124817, -0.00804087519645691, -0.015888655558228493, 0.0463591068983078, -0.01535851415246725, 0.0026698545552790165, 0.056394197046756744, 0.04553753882646561, 0.04237394407391548, -0.07591424137353897, 0.11570180207490921, 0.03696199133992195, 0.00035736511927098036, 0.05908538028597832, 0.08268771320581436, -0.024109698832035065, 0.048956628888845444, -0.06876591593027115, -3.833338269032538e-05, 0.09647155553102493, -0.08054504543542862, -0.0955846831202507, -0.0017027535941451788, -0.04785635694861412, -0.05418204888701439, 0.028477463871240616, 0.03480992466211319, 0.051497578620910645, 0.023661645129323006, 0.06537220627069473, 0.027242416515946388, 0.01491440087556839, 0.10259571671485901, 0.07398367673158646, 0.0038263320457190275, -0.019927863031625748, -0.051859963685274124, 0.021298065781593323, 0.042290396988391876, 0.016453146934509277, -0.05092667415738106, 0.05296962708234787, 0.021308505907654762, -0.02866007387638092, 0.03897630050778389, 0.04403326287865639, 0.018725648522377014, -0.03732626140117645, 0.004915689583867788, -0.03208749741315842, 0.027658428996801376, -0.05529087781906128, 0.003782506799325347, -0.07088987529277802, -0.021565265953540802, -0.017706969752907753, 0.0007037647883407772, -0.0844217911362648, -0.09855952858924866, -0.07291657477617264, 0.044099222868680954, 0.06036465987563133, 0.048204995691776276, -0.08975778520107269, -0.07571445405483246, 0.0675772875547409, -0.023358628153800964, -0.008089416660368443, 0.04744209721684456, -0.013870041817426682, 0.01786457560956478, -0.002981336321681738, -0.05356410890817642, 0.08641795068979263, -0.07299888134002686, 0.06158272176980972, 0.04357645660638809, 0.005281088873744011, -0.027776002883911133, 0.09150253236293793, -0.0860871970653534, 0.0548531711101532, 0.03501208499073982, 0.042785756289958954, 0.0922185480594635, -0.015300747007131577, 0.05320955067873001, -0.01156834326684475, 0.009222549386322498, -0.059514664113521576, -0.04875923693180084, -0.06434483826160431, 0.06269095093011856, 0.07235509902238846, 0.02137898840010166, 0.07045331597328186, -0.06971486657857895, 0.028940247371792793, -0.00803141389042139, 0.0023838700726628304, 0.047017112374305725, 0.04238910600543022, 0.06170709803700447, 0.04528787359595299, 0.052916303277015686, -0.05683435872197151, -0.042620472609996796, -0.006383092142641544, -0.013270536437630653, 0.09854539483785629, 0.10562462359666824, 0.022713588550686836, -0.011556304059922695, -0.03053227812051773, 0.033371903002262115, 0.09790118783712387, -0.051745831966400146, 0.06759617477655411, -0.030595935881137848, 0.006384297274053097, -0.016049258410930634, -0.08502420037984848, -0.010097733698785305, 0.04761102795600891, -0.07982689142227173, -0.012915175408124924, 0.019039617851376534, 0.09922059625387192, -0.04039250314235687, 0.10848499834537506, -0.04189790040254593, 0.02676602639257908, -0.02028937079012394, -0.051091257482767105, -0.06179162114858627, -0.06484956294298172, 0.05131027102470398, 0.00460826838389039, -0.007079899311065674, 0.030943773686885834, 0.0015034587122499943, -0.015168377198278904, -0.003168976865708828, 0.024759981781244278, 0.019561832770705223, -0.052496250718832016, -0.08169514685869217, -0.0065470472909510136, 0.07192867994308472, 0.06266357004642487, -0.06870874017477036, -0.10098756104707718, 0.009706526063382626, -0.003360156901180744, -0.036570243537425995, 0.01685107871890068, 0.08235479891300201, 0.0799369364976883, 0.030288223177194595, 0.01745699532330036, 0.000243851711275056, 0.051625099033117294, -0.05644966661930084, -0.04336201772093773, 0.03130641207098961, 0.017386801540851593, 0.022930629551410675, 0.04147164151072502, 0.04947284981608391, -0.02799941971898079, -0.09029562026262283, -0.00361522170715034, -0.07426387816667557, -0.03450943902134895, -0.0394204780459404, 0.007492888253182173, 0.0680762529373169, -0.04857206717133522, -0.06654573976993561, -0.025406159460544586, 0.03242170065641403, -0.02341596968472004, -0.0037217712961137295, -0.019425980746746063, 0.0021710083819925785, 0.05016486719250679, -0.04402005672454834, 0.013334497809410095, -0.0863296240568161, -0.017284052446484566, -0.07140190154314041, -0.07646487653255463, 0.10485326498746872, -0.031365394592285156, -0.050610341131687164, 0.04821256548166275, -0.04870990291237831, 0.009932211600244045, 0.014329435303807259, 0.02233942039310932, -0.0880589410662651, 0.06114779785275459, -0.08688446879386902, 0.030447212979197502, -0.04977714642882347, 0.07020238786935806, -0.0668870136141777, -0.04562634974718094, -0.05145551264286041, -0.01803685910999775, 0.022046856582164764, -0.0633407011628151, -0.03089115209877491, -0.05944986268877983, -0.05020495876669884, 0.06983844190835953, 0.02057858370244503, -0.037292931228876114, -0.00189649045933038, -0.06250453740358353, 0.0027864775620400906, 0.031047452241182327, -0.04224341735243797, 0.04270455986261368, -0.05221356824040413, -0.008428297005593777, -0.0711696520447731, -0.07491914927959442, 0.0151476189494133, -0.06407102197408676, 0.02304506115615368, 0.0478779636323452, -0.05697030574083328, 0.046962566673755646, -0.0601121187210083, -0.02197154238820076, 0.021472426131367683, 0.024578528478741646, -0.06776580959558487, -0.07983847707509995, 0.06653426587581635, 0.005568406078964472, -0.04231489077210426, -0.09179453551769257, -0.010912914760410786, 0.00936660636216402, -0.06064606457948685, 0.060708463191986084, 0.07604648917913437, -0.02505565993487835, 0.014937174506485462, 0.03651837259531021, -0.052215736359357834, -0.026277652010321617, -0.10802190750837326, 0.03589595481753349, 0.062280457466840744, 0.038366321474313736, 0.021073883399367332, 0.025289595127105713, -0.050040312111377716, 0.07403157651424408, -0.09243554621934891, 0.023577649146318436, -0.0031366399489343166, -0.04640961438417435, 0.10801520943641663, -0.05353441834449768, -0.04445665702223778, -0.016480734571814537, -0.05839357525110245, 0.013208728283643723, 0.010535454377532005, 0.026032589375972748, -0.04106689989566803, -0.07317967712879181, 0.021817564964294434, -0.03700310364365578, -0.03679841384291649, 0.009697481989860535, 0.017711099237203598, 0.03691697493195534, 0.022353215143084526, 0.023925980553030968, -0.03960133343935013, -0.09117912501096725, 0.004900554195046425, 0.06518025696277618, 0.029704196378588676, -0.047306448221206665, 0.05834947153925896, 0.0052000004798173904, 0.030990220606327057, 0.019612593576312065, -0.006874583661556244, -0.08217772096395493, -0.0957910567522049, -0.014640066772699356, 0.09069687128067017, 0.05929553881287575, -0.0693550705909729, 0.02014799974858761, 0.00888180360198021, 0.07157589495182037, 0.07451024651527405, -0.00903757568448782, -0.013534162193536758, -0.11047720909118652, 0.03469961881637573, -0.03889711573719978, 0.052196353673934937, 0.005850409623235464, -0.018390396609902382, -0.05260217934846878, -0.008924959227442741, -0.05852751061320305, 0.0461294911801815, 0.00946086272597313, 0.05309927091002464, -0.0450555644929409, -0.06636281311511993, -0.04925045743584633, 0.07106669247150421, -0.05137907341122627, -0.05661945790052414, -0.002902443753555417, -0.0347401462495327, 0.08131968975067139, 0.01811371184885502, 0.02376064844429493, 0.02963402308523655, 0.07727847993373871, -0.05326901003718376, 0.044459257274866104, 0.02053261175751686, -0.09534020721912384, -0.03980035334825516, 0.08077828586101532, 0.045095738023519516, 0.05031920596957207, 0.05498340725898743, -0.07298564165830612, 0.04246117174625397, -0.10635767877101898, 0.020215628668665886, 0.08732885122299194, -0.056914277374744415, -0.08176379650831223, -0.03480246290564537, 0.052603501826524734, -0.09059218317270279, -0.05422963201999664, -0.008056874386966228, 0.09039810299873352, 0.019489528611302376, -0.0802285447716713, -0.008187953382730484, -0.040401361882686615, -0.061392251402139664, 0.0633060485124588, 0.08276019990444183, -0.08253172785043716, 0.05928884446620941, -0.06558561325073242, 0.03441817685961723, 0.035830892622470856, 0.05727402865886688, -0.06012085825204849, 0.0804448202252388, 0.09385714679956436, 0.04525364562869072, -0.03258227929472923, 0.04279521107673645, 0.06545227766036987, 0.06794457137584686, -0.08250395208597183, -0.11775874346494675, 0.06159993261098862, 0.06415542960166931, 0.08393115550279617, 0.04578448832035065, -0.03532000258564949, -0.048874661326408386, 0.07274960726499557, -0.05253075435757637, -0.016538862138986588, 0.06611143052577972, 0.024531586095690727, 0.04182809218764305, -0.07387451082468033, 0.05177682265639305, 0.054413918405771255, 0.07830823957920074, -0.05322222784161568, 0.019273243844509125, 0.07211746275424957, 0.05999898910522461, 0.004892205353826284, 0.014794240705668926, 0.005620178766548634, -0.014191217720508575, -0.038104258477687836, 0.028585856780409813, -0.020654061809182167, 0.04792464151978493, 0.025790724903345108, 0.09517533332109451, 0.05697265639901161, 0.03555740788578987, 0.05208398401737213, -0.07700411975383759, -0.08427023887634277, -0.028963204473257065, 0.02719908393919468, -0.03237374499440193, -0.08184255659580231, 0.00854402408003807, 0.06361605226993561, 0.05746648088097572, -0.04499020799994469, 0.07488087564706802, 0.004704025108367205, 0.02197217009961605, -0.012699797749519348, -0.02438146062195301, 0.09632056951522827, 0.0332036092877388, -0.053993962705135345, -0.05771264061331749, -0.0602341964840889, -0.009691881947219372, 0.06748593598604202, -0.08077910542488098, 0.04801897704601288, -0.024184228852391243, 0.03378446027636528, -0.05384872481226921, -0.05998890474438667, 0.007738201878964901, 0.060518670827150345, -0.0634022057056427, -0.07179652899503708, -0.06749837845563889, 0.03080410324037075, 0.06430800259113312, -0.004885422065854073, 0.0034227310679852962, 0.05103960633277893, 0.018356608226895332, 0.021487031131982803, -0.08698992431163788, 0.04624369740486145, 0.02749985083937645, 0.008970054797828197, 0.03189989551901817, 0.0002504207077436149, 0.03465370833873749, 0.06379328668117523, 0.003988437820225954, -0.0773974284529686, -0.006812932901084423, -0.0014846068806946278, -0.017696985974907875, -0.020720958709716797, 0.07789649814367294, -0.02677563950419426, -0.034326110035181046, -0.02035585418343544, 0.023808026686310768, 0.09569824486970901, -0.011851103976368904, -0.021108273416757584, -0.0034466409124433994, 0.00910139363259077, -0.05040939152240753, 0.07290302217006683, 0.0013634241186082363, -0.022268425673246384, -0.07523076236248016, 0.024877723306417465, -0.04423125088214874, -0.04518083855509758, 0.010578373447060585, -0.09929712861776352, 0.04657943546772003, 0.024515310302376747, 0.03463445603847504, -0.0022019429598003626, 0.06015723943710327, 0.0044614640064537525, 0.03357702121138573, -0.05030831694602966, 0.06365393847227097, -0.044579651206731796, -0.013830314390361309, 0.10071447491645813, -0.002057804260402918, -0.05721884220838547, 0.001871590269729495, -0.02199525199830532, 0.013507080264389515, 0.0507240891456604, 0.05158344283699989, 0.006691087037324905, -0.09448683261871338, -0.07928043603897095, 0.01206226833164692, 0.05571559816598892, 0.06874685734510422, -0.01827976480126381, -0.11365005373954773, -0.01705074869096279, 0.038319237530231476, -0.04117479547858238, -0.10281137377023697, 0.05471610650420189, 0.05995814502239227, 0.0064537725411355495, -0.032524533569812775, 0.05414858087897301, 0.025558222085237503, 0.0006342900451272726, 0.030370188876986504, 0.0731976181268692, -0.10032020509243011, 0.02580835297703743, -0.005505451932549477, -0.09537304192781448, 0.0449320450425148, -0.030181284993886948, -0.06624776124954224, -0.05477053299546242, -0.033476572483778, -0.035941511392593384, -0.02044849842786789, 0.009963070042431355, -0.05624353513121605, -0.009144061245024204, 0.006480369716882706, 0.05476009473204613, -0.10044199973344803, 0.04654061794281006, -0.0710797980427742, 0.0009305146522819996, 0.0004804132622666657, -0.0588652603328228, 0.016769137233495712, 0.05194328725337982, 0.028143098577857018, 0.05120709538459778, 0.03852338716387749, -0.06828135251998901, 0.08967988193035126, -0.012479201890528202, 0.07516486942768097, -0.0660950168967247, -0.006229581777006388, -0.017538584768772125, 0.03956850245594978, -0.014211599715054035, 0.02060745470225811, 0.07040128856897354, -0.01305180974304676, 0.01696707122027874, 0.027644312009215355, -0.07688897103071213, -0.05767601728439331, 0.008464754559099674, 0.07389220595359802, 0.007329998537898064, 0.07459016889333725, -0.06563276052474976, 0.027719056233763695, -0.05878382921218872, 0.051019057631492615, -0.06067923456430435, -0.04808693379163742, 0.0940215066075325, -0.06330521404743195, 0.04072264954447746, -0.08695089817047119, -0.10064475238323212, -0.054007645696401596, -0.010943068191409111, -0.02775234915316105, -0.07624319940805435, -0.05277558043599129, -0.053044307976961136, -0.025280145928263664, 0.018381314352154732, 0.06551507115364075, 0.08391604572534561, -0.08228128403425217, -0.03819943964481354, -0.009797654114663601, -0.0894039049744606, -0.08612029254436493, -0.07203661650419235, 0.024341171607375145, 0.019424445927143097, -0.010292948223650455, -0.032958973199129105, 0.054633885622024536, -0.07380571961402893, -0.068013034760952, 0.023876141756772995, 0.06435732543468475, -0.10018479824066162, -0.07536489516496658, -0.07766093313694, 0.013345684856176376, 0.016991285607218742, -0.06419096142053604, -0.004760297480970621, -0.007067909464240074, 0.07641523331403732, 0.016686273738741875, -0.05539214611053467, -0.05980286747217178, 0.0196804441511631, 0.040891021490097046, 0.020643768832087517, 0.0035308378282934427, 0.014666212722659111, 0.0321354866027832, -0.03918797895312309, -0.026371333748102188, 0.03871767595410347, -0.039034154266119, -0.05060002952814102, -0.018679849803447723, -0.06087269261479378, 0.07552796602249146, -0.049360815435647964, 0.04697611182928085, -0.03352462872862816, -0.04935565963387489, -0.020961953327059746, 0.04818808659911156, 0.026459358632564545, 0.006493506021797657, 0.06930261105298996, 0.05615437403321266, 0.0242487620562315, 0.09043549746274948, -0.01452766265720129, 0.03179290145635605, 0.0306721068918705, 0.08280199021100998, 0.09014690667390823, -0.02593943476676941, 0.07202490419149399, -0.0785394236445427, -0.02052217349410057, -0.05340751260519028, 0.054143089801073074, 0.06390257179737091, 0.03757443279027939, -0.07171536237001419, 0.0013869842514395714, 0.03090137615799904, 0.06437411904335022, -0.016499198973178864, -0.01297028362751007, 0.008477230556309223, 0.04344313219189644, 0.055471841245889664, -0.09351705014705658, 0.02956360951066017, -0.012239894829690456, -0.07384885847568512, 0.04455220699310303, 0.06558780372142792, 0.044776927679777145, 0.05624661594629288, 0.0061594597063958645, 0.08108217269182205, 0.011568006128072739, -0.02223442681133747, -0.03093867003917694, -0.04068838432431221, 0.06272357702255249, -0.023645037785172462, 0.002693680813536048, -0.09933798015117645, -0.018258752301335335, -0.033206287771463394, 0.03371213376522064, -0.0489204004406929, -0.05328667536377907, -0.002285798778757453, -0.07398959994316101, 0.01651487872004509, 0.0472845733165741, 0.05236951261758804, -0.06867796927690506, 0.09951680153608322, -0.06775284558534622, 0.10350953042507172, -0.07109438627958298, -0.06471652537584305, 0.001464643981307745, -0.056504976004362106, 0.042645763605833054, -0.026737337931990623, 0.07778632640838623, -0.05654764547944069, -0.02480986714363098, -0.021301988512277603, 0.0819973424077034, 0.056566037237644196, 0.03462773934006691, -0.00882359966635704, -0.06475897878408432, 0.09333085268735886, -0.04291863739490509, -0.013006843626499176, -0.05645314231514931, -0.055805932730436325, -0.035117559134960175, -0.026008402928709984, -0.0016339188441634178, 0.05904129520058632, 0.06462099403142929, 0.008653420954942703, 0.004996781703084707, -0.06541191786527634, -0.03447392210364342, -0.08100008219480515, -0.0381302647292614, 0.024003738537430763, 0.034712351858615875, 0.0456014908850193, -0.10012496262788773, -0.08917702734470367, -0.015508584678173065, 0.05920753628015518, 0.031108038499951363, 0.0006704777479171753, -0.04152647405862808, -0.0008279175963252783, 0.08129142969846725, 0.0732838585972786, -0.055598221719264984, 0.061347395181655884, -0.03739648684859276, -0.0285189189016819, -0.02850758470594883, -0.085545614361763, 0.010972410440444946, -0.0951395258307457, -0.0036803025286644697, -0.06697915494441986, -0.027366776019334793, 0.08506234735250473, -0.09933960437774658, -0.08896368741989136, 0.02970665693283081, -0.015961844474077225, -0.07311049848794937, -0.06761340796947479, 0.08073253929615021, -0.11248422414064407, -0.005835545249283314, -0.05525466427206993, -0.08605516701936722, -0.0803891122341156, 0.009783382527530193, -0.058299534022808075, -0.007239197380840778, 0.04373927786946297, -0.06228802725672722, -0.07318521291017532, -0.06612016260623932, 0.018740346655249596, -0.015634212642908096, 0.0010744498576968908, 0.0578143447637558, -0.014443997293710709, -0.10732416063547134, -0.07257423549890518, -0.06067011132836342, 0.06027542054653168, 4.6851433580741286e-05, 0.04197227954864502, 0.0020605893805623055, -0.059592604637145996, 0.026716401800513268, -0.08885067701339722, -0.032317813485860825, 0.03423710912466049, -0.021807799115777016, 0.030700551345944405, -0.09735080599784851, 0.0017588055925443769, 0.006945296656340361, -0.03356876224279404, 0.06736799329519272, -0.04825793206691742, 0.02029530704021454, -0.04104531556367874, 0.04606819152832031, -0.09164591133594513, 0.024781422689557076, -0.0410289391875267, 0.00171937164850533, -0.05308596044778824, 0.004618125036358833, -0.017419682815670967, -0.023007875308394432, -0.01013980619609356, -0.00520667340606451, -0.01191882323473692, -0.008432751521468163, 0.06651900708675385, -0.02645677700638771, -0.06091581657528877, -0.004326626658439636, -0.0666675865650177, 0.014942631125450134, 0.03794584795832634, -0.08648326247930527, -0.07190433144569397, 0.07515691220760345, 0.0648069977760315, 0.09019289165735245, -0.11930251121520996, -0.015068333595991135, -0.10626915097236633, 0.03586116060614586, -0.05615891516208649, 0.03534131124615669, -0.05298268049955368, 0.017112847417593002, -0.027983445674180984, 0.0005309288389980793, 0.039773520082235336, -0.07060152292251587, -0.06040222942829132, -0.02108069695532322, 0.0008561427239328623, 0.012186616659164429, -0.03863057494163513, -0.07462538033723831, -0.05605524033308029, -0.10760203748941422, 0.04043148458003998, 0.03620145469903946, -0.01505700871348381, -0.029881354421377182, -0.05077049136161804, -0.10297077894210815, 0.007974009029567242, -0.004157559480518103, -0.06677036732435226, -0.024417100474238396, -0.005404932890087366, -0.0013654414797201753, 0.08240737020969391, 0.08634816855192184, -0.01594691351056099, 0.06264164298772812, 0.06355292350053787, 0.048326730728149414, 0.01475229486823082, 0.047992605715990067, 0.05004680156707764, 0.04558512941002846, -0.020547926425933838, -0.052374858409166336, 0.005784272216260433, -0.04026985168457031, 0.033472687005996704, 0.008963351137936115, -0.04175703227519989, 0.025797590613365173, 0.0630330815911293, 0.06907284259796143, 0.044745661318302155, -0.015184396877884865, -0.018545236438512802, 0.07160203158855438, 0.04214024916291237, 0.0015995381399989128, -0.08520844578742981, 0.024841640144586563, 0.0323425717651844, -0.05719689279794693, 0.04659464955329895, -0.0719522163271904, 0.03051534853875637, 0.028395095840096474, -0.03393690660595894, -0.011038734577596188, 0.03664587438106537, 0.008660221472382545, 0.07058139890432358, -0.029357818886637688, -0.07102265954017639, -0.0075453403405845165, 0.044045232236385345, 0.05567500740289688, -0.03581889346241951, -0.0039091878570616245, 0.04474969208240509, 0.06118379533290863, 0.0713406577706337, 0.09319071471691132, -0.04809494689106941, -0.02798394300043583, 0.08874449878931046, -0.05377134308218956, -0.022873559966683388, 0.05533255264163017, 0.029971694573760033, -0.06549067795276642, 0.010753300040960312, -0.028933262452483177, -0.09780597686767578, -0.04038120061159134, 0.043899234384298325, -0.08513538539409637, -0.06768333166837692, -0.06115635111927986, -0.055103857070207596, 0.0716010332107544, 0.04544876143336296, 0.02452712506055832, 0.09249813854694366, -0.027455640956759453, 0.017741786316037178, -0.00016229136963374913, 0.07244111597537994, 0.0813964456319809, -0.007804725784808397, 0.009873826056718826, 0.03709607198834419, 0.04157061129808426, -0.017635541036725044, -0.0425257682800293, 0.06556019186973572, 0.0082669323310256, 0.04030410200357437, 0.030658811330795288, 0.038663286715745926, 0.07041005790233612, 0.051155220717191696, 0.045589178800582886, 0.020162835717201233, 0.05447405204176903, 0.05786283686757088, -0.07361552119255066, -0.08441327512264252, -0.07542389631271362, -0.05895654112100601, 0.07718569040298462, -0.003044105600565672, 0.03668273240327835, -0.012633297592401505, 0.05150241404771805, 0.013610455207526684, -0.05336560681462288, 0.011439470574259758, 0.027564790099859238, 0.08430516719818115, -0.05504564568400383, 0.08478305488824844, -0.04408072307705879, -0.06643975526094437, -0.033128343522548676, -0.012814588844776154, -0.06693987548351288, 0.08574368059635162, -0.024447910487651825, 0.07008754462003708, -0.05328252166509628, -0.03262469172477722, -0.01895192824304104, -0.014448780566453934, 0.008449461311101913, 0.0815146192908287, 0.023651037365198135, -0.04973849654197693, -0.04434768855571747, 0.016044963151216507, 0.05460326373577118, -0.034679267555475235, 0.0995337963104248, 0.02080238051712513, 0.0310163926333189, 0.0775226354598999, 0.04873702675104141, 0.07227639108896255, 0.06743741780519485, -0.02142265997827053, -0.014989912509918213, 0.07939116656780243, 0.07004795968532562, 0.09375303983688354, 0.011175692081451416, -0.07589562237262726, -0.0458722747862339, 0.0658351480960846, 0.0037650533486157656, -0.05039699748158455, -0.049384187906980515, 0.0670689046382904, 0.03378313407301903, -0.06903724372386932, 0.04784628003835678, -0.010304858908057213, 0.024828100576996803, 0.07020099461078644, -0.07836118340492249, 0.08476416766643524, 0.08138318359851837, -0.016749141737818718, -0.051379017531871796, -0.07348355650901794, -0.0945870652794838, -0.0510735809803009, 0.09530074894428253, -0.04548227787017822, 0.01984351873397827, -0.07922052592039108, -0.07943151146173477, 0.05121731013059616, 0.032328683882951736, -0.025189321488142014, 0.005065928213298321, -0.023426160216331482, -0.10669387131929398, 0.07009115070104599, -0.050818514078855515, -0.028484536334872246, 0.08142538368701935, -0.014641713351011276, -0.08357366919517517, 0.017142070457339287, 0.02978770062327385, -0.12364159524440765, 0.05283329263329506, -0.015276145190000534, -0.05487692356109619, 0.05520014092326164, -0.0923253521323204, -0.0373404324054718, -0.06663577258586884, 0.007145852316170931, -0.057119615375995636, -0.048708781599998474, -0.02924109250307083, 0.024260766804218292, -0.072477325797081, 0.004263802897185087, -0.00694655766710639, -0.02083604596555233, -0.05728737637400627, -0.020190568640828133, 0.06177832558751106, 0.06750956177711487, 0.09557722508907318, 0.00808775145560503, 0.08435826003551483, -0.013866188935935497, 0.004434524103999138, -0.033499956130981445, -0.05863957107067108, -0.03154550865292549, -0.08321286737918854, -0.07397974282503128, 0.08015164732933044, -0.0191411841660738, -0.0259656123816967, 0.08887587487697601, 0.05879439041018486, 0.09650211036205292, 0.028315337374806404, -0.020558396354317665, 0.05016837269067764, -0.030127782374620438, 0.05460583418607712, -0.06461755186319351, -0.11731478571891785, 0.0651138573884964, 0.08966223895549774, -0.06369071453809738, -0.04024118930101395, -0.05550263822078705, 0.01072443276643753, 0.054551396518945694, -0.07502322643995285, -0.011757321655750275, -0.0024602676276117563, 0.0717606320977211, 0.04411693662405014, -0.048976924270391464, 0.05082673206925392, 0.07046295702457428, -0.07744535058736801, -0.017261791974306107, -0.005847565364092588, -0.0010159171652048826, -0.0478774756193161, -0.04040305316448212, -0.005433633457869291, 0.0478658601641655, -0.00475161150097847, -0.01532828900963068, -0.012343114241957664, -0.10014205425977707, -0.09435366839170456, -0.04365570470690727, 0.07201547175645828, -0.02989402413368225, 0.08548758178949356, 0.032804492861032486, 0.06333598494529724, -0.056398142129182816, 0.038169294595718384, -0.06365934759378433, -0.0012075189733877778, 0.06012147292494774, 0.040860358625650406, 0.031942613422870636, -0.06068982928991318, -0.06160448119044304, 0.05439244210720062, 0.05485572665929794, 0.029460709542036057, -0.019714372232556343, -0.009099572896957397, -0.020788267254829407, -0.06369877606630325, 0.0003414032398723066, 0.09641559422016144, 0.08274386078119278, -0.02096470445394516, -0.02712908200919628, -0.07246845960617065, -0.0056787827052176, 0.005148138850927353, -0.006052091252058744, -0.06383375823497772, -0.04254700616002083, -0.017866555601358414, -0.012532461434602737, 0.012020330876111984, -0.027785860002040863, 0.06720341742038727, 0.06074546277523041, -0.05436099320650101, -0.06368719041347504, -0.05896282196044922, 0.019108086824417114, 0.06387818604707718, -0.01656227558851242, -0.08065614104270935, -0.04671839624643326, 0.005281010642647743, -0.06865477561950684, -0.037073053419589996, -0.07327225059270859, -0.001050331280566752, 0.05925849825143814, -0.10928989946842194, -0.0018049099016934633, -0.0776156485080719, 0.002108696848154068, 0.005289538763463497, -0.06886573880910873, -0.0175011046230793, -0.05657188221812248, 0.021120503544807434, 0.026097126305103302, 0.00044600872206501663, -0.08928043395280838, -0.06856285780668259, -0.04344852641224861, -0.03463510423898697, -0.014439002610743046, 0.06282553821802139, 0.014300608076155186, -0.112770214676857, 0.02552594430744648, -0.10363466292619705, 0.05361064150929451, 0.0307825468480587, -0.011523677036166191, -0.058510683476924896, -0.03965282812714577, -0.10666444152593613, 0.04557495936751366, -0.05276141315698624, -0.03852824121713638, -0.11642368882894516, -0.0505291149020195, -0.0323503352701664, -0.030448943376541138, -0.04830491915345192, -0.054898377507925034, 0.013507555238902569, -0.016671892255544662, -0.05205950513482094, 0.07178942859172821, 0.03586871549487114, -0.03457259386777878, 0.051765162497758865, -0.048223696649074554, 0.005850804504007101, -0.061702240258455276, -0.11794096976518631, -0.050569161772727966, -0.09320779889822006, -0.025307178497314453, 0.029137467965483665, 0.002688125241547823, -0.0346403494477272, 0.01921180821955204, -0.024445107206702232, -0.10321520268917084, -0.004875193815678358, -0.07144396752119064, -0.029430773109197617, 0.03697187826037407, -0.08880544453859329, 0.05250604450702667, -0.07319626957178116, -0.12955451011657715, -0.049172718077898026, -0.024750441312789917, -0.06500588357448578, 0.015898022800683975, 0.0842212662100792, -0.06485748291015625, -0.04788319393992424, -0.022069647908210754, -0.01620239019393921, -0.06206901744008064, -0.07811354100704193, -0.03074001893401146, -0.04313386604189873, -0.045784108340740204, 0.01807623915374279, 0.028303634375333786, 0.028346911072731018, 0.03798821195960045, -0.03773743286728859, -0.005273551680147648, -0.017748907208442688, 0.1062123253941536, -0.07267403602600098, -0.07674239575862885, -0.03229396417737007, -0.033059220761060715, -0.017486747354269028, 0.0355977937579155, 0.02735684998333454, -0.11034786701202393, -0.0586945116519928, 0.026791347190737724, -0.06673316657543182, 0.036590442061424255, 0.036324046552181244, 0.005493188742548227, -0.06985479593276978, 0.026602763682603836, -0.010731234215199947, -0.014910971745848656, -0.011609304696321487, -0.00035802737693302333, 0.1062549278140068, 0.09465160220861435, 0.05192149803042412, -0.14193490147590637, -0.09567756950855255, -0.022872278466820717, 0.02285595051944256, -0.05005228891968727, 0.0775955468416214, 0.06137131154537201, 0.0987745076417923, 0.08165822923183441, -0.06193525344133377, -0.008787360973656178, -0.06301984190940857, -0.09780386090278625, -0.09609072655439377, 0.05325173959136009, -0.09220009297132492, 0.07252509891986847, -0.031199989840388298, 0.01181753445416689, -0.06293249130249023, -0.03345314413309097, -0.032215867191553116, -0.008824937976896763, -0.05588097125291824, -0.049673669040203094, 0.0744517371058464, 0.06295783072710037, -0.057415030896663666, 0.022374672815203667, -0.09056021273136139, 0.031418442726135254, -0.10550551116466522, 0.06919785588979721, 0.09510571509599686, -0.041966937482357025, 0.036364682018756866, 0.04121607914566994, -0.04111775755882263, -0.04492026939988136, -0.038512539118528366, 0.031051030382514, -0.057881906628608704, -0.07154317200183868, -0.0804651752114296, 0.0324617438018322, 0.005165360402315855, -0.008073057048022747, -0.08013002574443817, 0.04403100535273552, 0.0015366520965471864, -0.0920264795422554, 0.03145892918109894, -0.025128228589892387, 0.05558866634964943, -0.038117386400699615, -0.03766125068068504, -0.05669661983847618, -0.029812153428792953, -0.0870511531829834, 0.06422831863164902, -0.027763182297348976, 0.004054689314216375, 0.06422668695449829, -0.0003552468551788479, -0.014186527580022812, -0.01234107930213213, -0.09790538251399994, -0.08350234478712082, 0.08919044584035873, 0.01643540896475315, 0.011799381114542484, -0.007184785325080156, 0.021173890680074692, 0.018902814015746117, -0.06317772716283798, 0.013905408792197704, -0.06470376253128052, -0.04318223148584366, -0.001506737433373928, 0.06734059751033783, 0.06442314386367798, 0.007816504687070847, -0.0588107667863369, 0.02046244405210018, 0.061836834996938705, -0.05776742473244667, -0.03247259557247162, 0.0635785385966301, -0.06944531947374344, -0.05133727192878723, 0.08017254620790482, -0.05236167088150978, 0.05499397590756416, 0.0665692463517189, -0.009734696708619595, -0.04713301733136177, 0.022143064066767693, -0.06976382434368134, -0.07150839269161224, -0.017441896721720695, -0.012178825214505196, -0.07364071160554886, 0.026765165850520134, 0.017459861934185028, 0.0091268764808774, 0.06765345484018326, 0.0622362457215786, -0.0234517902135849, -0.10324157774448395, 0.06436488777399063, 0.06009884551167488, -0.043746352195739746, -0.006376858800649643, 0.011863991618156433, -0.03652426227927208, 0.08972274512052536, -0.041954848915338516, 0.0391249842941761, 0.03395029157400131, 0.08790719509124756, -0.0706610232591629, 0.06799188256263733, 0.04903935268521309, -0.00012932735262438655, -0.04763453081250191, -0.03327557444572449, 0.057830169796943665, 0.039299558848142624, 0.03331586346030235, 0.02305877022445202, 0.06214999035000801, 0.054549455642700195, -0.0641128197312355, 0.049367520958185196, 0.003171188523992896, -0.02321397326886654, 0.09746474027633667, -0.04957175254821777, 0.05722029134631157, 0.025118347257375717, -0.0016177201177924871, -0.015795793384313583, -0.02896660566329956, 0.0293185506016016, -0.04130714759230614, 0.0486120842397213, 0.07584290206432343, -0.021526355296373367, -0.0643005445599556, 0.07984575629234314, -0.06446849554777145, 0.01227351650595665, -0.050227414816617966, -0.04214023798704147, 0.006936999037861824, -0.019089804962277412, -0.1147460788488388, -0.013540945015847683, -0.06197110190987587, -0.014676885679364204, -0.04635270684957504, 0.06387963891029358, 0.08588997274637222, 0.03850869834423065, 0.09337083995342255, 0.0889052152633667, 0.025589950382709503, 0.0012493433896452188, -0.052148282527923584, -0.058567341417074203, -0.00924665480852127, 0.00865735299885273, 0.04496514052152634, 0.05105443671345711, 0.03512479364871979, 0.052222464233636856, 0.050736334174871445, -0.01272426825016737, -0.056954264640808105, 0.044901374727487564, -0.09406652301549911, -0.0155909713357687, 0.059081368148326874, 0.026041312143206596, -0.013571630232036114, 0.022956423461437225, 0.031251341104507446, -0.010133426636457443, -0.0006635947502218187, -0.08677849173545837, 0.050270527601242065, -0.009805893525481224, -0.08530735224485397, 0.0808892548084259, 0.0758204311132431, -0.07782763987779617, 0.03460436686873436, -0.0067110825330019, 0.02571539767086506, 0.04863828793168068, 0.07483044266700745, -0.018183859065175056, 0.0592159666121006, -0.030991487205028534, 0.08796628564596176, 0.008678956888616085, 0.03048885241150856, -0.07793295383453369, -0.054630327969789505, -0.04594351723790169, -0.04921694099903107, 0.014178968034684658, -0.07969372719526291, -0.022026289254426956, -0.05979597568511963, -0.05382566526532173, -0.0610620379447937, -0.06608138978481293, -0.07350239902734756, 0.002010862808674574, -0.015482576563954353, 0.014121562242507935, -0.09190841019153595, 0.031383369117975235, 0.0817679613828659, 0.016547828912734985, -0.08314014971256256, 0.0516398660838604, -0.053151149302721024, -0.008957920596003532, 0.0014733016723766923, 0.009877484291791916, -0.022165611386299133, -0.09230327606201172, -0.030613739043474197, -0.046979330480098724, -0.027271650731563568, -0.049251243472099304, -0.002523093717172742, -0.09382003545761108, -0.020218078047037125, 0.11027936637401581, 0.01746261864900589, -0.011979153379797935, -0.057426098734140396, 0.058913297951221466, -0.03827352821826935, -0.025277605280280113, -0.07625288516283035, -0.0028292895294725895, -0.016939476132392883, -0.06282974034547806, 0.032144736498594284, 0.09871429204940796, -0.04303125664591789, 0.020774319767951965, -0.06496010720729828, -0.04282098263502121, -0.08530787378549576, 0.03576561063528061, -0.005152835510671139, -0.019238611683249474, 0.030775325372815132, -0.058828745037317276, 0.06229202076792717, -0.06393589079380035, 0.08758310228586197, 0.058825504034757614, 0.07829955220222473, -0.011917266063392162, 0.024161498993635178, 0.02920990251004696, 0.045586030930280685, -0.04805297777056694, -0.024016419425606728, -0.057699188590049744, 0.03914671391248703, 0.011343522928655148, 0.03547840565443039, -0.09513390064239502, 0.029665006324648857, -0.026673059910535812, 0.0532473586499691, 0.07613217830657959, 0.08185142278671265, 0.059468913823366165, 0.055947400629520416, -0.007733858190476894, 0.04998457059264183, -0.042553868144750595, -0.060202669352293015, 0.05736136436462402, -0.019252024590969086, -0.07291827350854874, -0.06848928332328796, 0.07245256751775742, -0.012174124829471111, 0.018458755686879158, 0.08554713428020477, -0.054906394332647324, -0.08180706948041916, -0.04981429874897003, -0.0544654056429863, 0.04008062928915024, 0.046224284917116165, -0.0751151442527771, 0.012966939248144627, 0.04622437804937363, -0.06250342726707458, 0.09010427445173264, -0.017397440969944, 0.07701535522937775, -0.0330975279211998, 0.003086625598371029, 0.00914955884218216, -0.005696161184459925, -0.004007034935057163, -0.01612609066069126, -0.030204622074961662, 0.06378631293773651, 0.0025613452307879925, 0.0014022414106875658, -0.052134618163108826, -0.03733835369348526, -0.03140324726700783, -0.025455817580223083, -0.07237516343593597, 0.09515146166086197, 0.030410876497626305, -0.07381134480237961, -0.0030426476150751114, 0.008030680939555168, 0.047153159976005554, -0.0685158297419548, 0.0893414095044136, 0.05237250030040741, 0.05737931281328201, 0.015319324098527431, 0.046555422246456146, 0.03561338782310486, 0.06563882529735565, -0.04867877811193466, 0.08280718326568604, 0.021477486938238144, -0.06988108158111572, 0.08463288843631744, 0.005487246438860893, -0.08256193995475769, -0.07607191056013107, -0.010775493457913399, 0.09227162599563599, -0.06994185596704483, -0.039411138743162155, 0.02923375740647316, 0.009802230633795261, 0.04411345720291138, -0.09519244730472565, 0.04974447190761566, -0.022216755896806717, -0.07186895608901978, -0.05334899201989174, -0.009321654215455055, 0.037424035370349884, 0.034870125353336334, -0.04733589291572571, 0.05192568898200989, 0.014488792046904564, -0.0010200589895248413, 0.020344501361250877, 0.07046610862016678, -0.04609794542193413, 0.046734098345041275, -0.06345362961292267, -0.06699574738740921, 0.04235763102769852, -0.028604703024029732, 0.05270371586084366, -0.0018602123018354177, 0.01782909780740738, -0.09841933101415634, 0.043820351362228394, -0.09058132022619247, -0.004204901400953531, 0.021994100883603096, -0.07242506742477417, 0.04509387165307999, 0.02734863944351673, -0.06641562283039093, 0.011498470790684223, -0.07463787496089935, -0.08610524237155914, 0.03221144899725914, -0.04745860770344734, 0.10208196192979813, -0.07363487780094147, 0.014267153106629848, -0.026799656450748444, 0.03420504182577133, -0.06857001781463623, 0.04919029027223587, -0.0014606432523578405, -0.05872371047735214, 0.08183173835277557, 0.016577735543251038, 0.06064798682928085, 0.0014788552653044462, -0.02887747250497341, 0.015643829479813576, 0.06700253486633301, 0.02365700900554657, 0.06408217549324036, -0.0988222137093544, -0.03896661102771759, -0.01455047819763422, 0.0029376258607953787, -0.04331653192639351, 0.07812125980854034, 0.013311527669429779, 0.06529192626476288, 0.01691851019859314, -0.038615889847278595, -0.07278843969106674, 0.062056414783000946, 0.015028012916445732, 0.06680013984441757, -0.04068370163440704, -0.056131552904844284, -0.0776049941778183, -0.05932921543717384, -0.08866357803344727, -0.038056064397096634, -0.030214719474315643, 0.03231394290924072, -0.08810970932245255, -0.0026419078931212425, -0.012051495723426342, 0.0528850182890892, -0.058090176433324814, 0.09589487314224243, 0.10212399810552597, -0.019977973774075508, -0.030962839722633362, -0.0010300028370693326, -0.08922142535448074, 0.05035904049873352, -0.07586228847503662, 0.005487595684826374, 0.03958868980407715, -0.09002364426851273, 0.07493533939123154, 0.019008755683898926, -0.0052012186497449875, 0.06159628555178642, 0.05196183919906616, -0.0694010853767395, 0.025968803092837334, 0.02113224007189274, -0.01788312941789627, -0.10816933214664459, -0.06807398051023483, 0.004504980985075235, -0.017063355073332787, -0.02708645537495613, 0.0323980413377285, -0.03395051509141922, -0.016978848725557327, -0.013463526964187622, -0.011619621887803078, 0.09375112503767014, 0.05667230486869812, -0.05446755513548851, 0.09520012140274048, -0.049843307584524155, -0.06438244134187698, -0.07229038327932358, -0.05299363285303116, 0.05013737082481384, 0.007122528739273548, 0.06400881707668304, 0.01508842408657074, 0.0012011543149128556, 0.013739325106143951, -0.044705841690301895, -0.019492579624056816, 0.01925469934940338, 0.0961664691567421, -0.04883624240756035, -0.07048622518777847, 0.059704940766096115, -0.036669548600912094, -0.06395974010229111, -0.03233828395605087, -0.031193755567073822, -0.08066868782043457, -0.007023974321782589, 0.0007911890861578286, 0.052693046629428864, 0.08029564470052719, -0.005621722899377346, -0.006225845776498318, 0.09256819635629654, -0.03337344154715538, 0.011042457073926926, 0.048323046416044235, -0.048048581928014755, 0.04289265349507332, -0.05254341661930084, -0.0452888049185276, -0.07419215887784958, -0.06741055101156235, -0.012237031012773514, -0.058240458369255066, -0.07499932497739792, 0.020646357908844948, -0.0460982508957386, 0.04221676290035248, -0.020394040271639824, -0.013005994260311127, 0.04537370055913925, -0.005214450415223837, -0.055348947644233704, 0.05933362618088722, 0.04293712228536606, -0.05050826445221901, 0.07907615602016449, -0.03080008178949356, -0.05854819342494011, -0.07120722532272339, -0.05265120789408684, 0.04768907278776169, 0.028663793578743935, -0.06606665253639221, -0.08690937608480453, -0.019167764112353325, 0.08612982928752899, -0.04984518140554428, 0.0063565559685230255, -0.04918096959590912, -0.018672090023756027, -0.05078772455453873, 0.006524206604808569, 0.06722079962491989, -0.011626459658145905, 0.02327624522149563, 0.0337509885430336, -0.09045145660638809, 0.04989400878548622, -0.05979132652282715, -0.06056107580661774, 0.09817161411046982, -0.06753984093666077, -0.014753613620996475, -0.042701613157987595, -0.044886264950037, -0.011379933916032314, 0.059868261218070984, 0.06288538873195648, 0.02373196929693222, -0.06851834803819656, -0.00477976631373167, 0.033630989491939545, 0.0046522729098796844, -0.03962697833776474, 0.06308148056268692, 0.07331295311450958, 0.06292938441038132, -0.03772170841693878, -0.08188634365797043, -0.013216119259595871, -0.018094230443239212, 0.0020979440305382013, -0.03503163531422615, 0.0946509912610054, -0.07532265037298203, 0.059275079518556595, -0.07368314266204834, 0.08237557113170624, 0.09699466079473495, -0.04753793403506279, 0.0738632082939148, -0.06709016859531403, 0.03765026852488518, 0.0516248382627964, 0.07762043923139572, -0.07171808183193207, 0.020223544910550117, -0.032518938183784485, -0.05764268711209297, -0.006470926571637392, -0.08333944529294968, 0.0417158342897892, -0.03273512050509453, 0.03651271015405655, 0.023226166144013405, -0.10684747248888016, -0.032978400588035583, -0.016430702060461044, 0.0446244552731514, 0.006562602706253529, 0.01846146024763584, 0.05081531032919884, 0.0664404109120369, -0.007543487939983606, 0.043340250849723816, -0.013438951224088669, -0.08734835684299469, -0.11575131118297577, -0.09541339427232742, 0.03189915046095848, -0.10456062853336334, 0.019250011071562767, -0.0024261788930743933, -0.07346202433109283, 0.06871291995048523, 0.05571497604250908, 0.06551160663366318, -0.053626056760549545, 0.015018543228507042, -0.10502807050943375, -0.02992931194603443, 0.10091656446456909, 0.018173977732658386, 0.07763068377971649, 0.06814418733119965, -0.09116759896278381, -0.05594396963715553, -0.009227982722222805, -0.019352849572896957, 0.0034575993195176125, -0.04188769310712814, -0.03902938589453697, 0.033951010555028915, -0.0012807807652279735, 0.0213201604783535, -0.07308733463287354, 0.07615139335393906, 0.09515821188688278, 0.055397018790245056, -0.06363126635551453, -0.07636482268571854, 0.006623866967856884, 0.0038531669415533543, -0.06860073655843735, 0.028438182547688484, 0.04263632372021675, -0.014344104565680027, -0.10336533188819885, -0.0812578946352005, 0.032799046486616135, -0.07232829183340073, 0.04443258419632912, -0.10623857378959656, -0.08110388368368149, 0.0879654660820961, 0.0695762187242508, -0.005722778383642435, -0.024964384734630585, 0.05714811384677887, 0.03520830348134041, 0.10569440573453903, 0.054049283266067505, -0.046706005930900574, 0.028936509042978287, 0.0339730829000473, -0.039440762251615524, -0.046289246529340744, -0.021188203245401382, 0.0336172878742218, 0.08939886093139648, -0.018284061923623085, -0.016041824594140053, 0.07116308063268661, -0.024987470358610153, -0.09898252785205841, 0.0238367710262537, 0.05935779586434364, 0.05980134755373001, 0.01498651783913374, 0.057496290653944016, -0.008476710878312588, -0.03437507897615433, 0.10697434097528458, 0.04790984466671944, -0.060671526938676834, -0.03729354217648506, -0.052398115396499634, -0.0247841514647007, -0.02795420028269291, 0.016665194183588028, -0.02102147415280342, -0.05978616327047348, 0.04432154819369316, -0.06431163102388382, -0.07209131866693497, 0.06528759002685547, 0.019209686666727066, 0.033987730741500854, -0.0007571607129648328, -0.027225641533732414, -0.032745834439992905, 0.06345372647047043, -0.05376117676496506, 0.07769060134887695, 0.10752850770950317, -0.03704800456762314, -0.021177947521209717, 0.039771173149347305, -0.07185309380292892, -0.0829065591096878, -0.05826110765337944, 0.07691145688295364, -0.009091544896364212, -0.04775708541274071, -0.044581495225429535, -0.05031725391745567, -0.07232839614152908, -0.018003111705183983, -0.022526854649186134, 0.05139373615384102, 0.09631384164094925, -0.03241594508290291, -0.07527288049459457, 0.007880431599915028, 0.03079628013074398, -0.06847379356622696, 0.010897716507315636, 0.05712870508432388, 0.07320623844861984, 0.07086718082427979, 0.04227181896567345, -0.030268598347902298, 0.03499067574739456, -0.07210007309913635, -0.016593273729085922, -0.11166229099035263, -0.0149888526648283, -0.026596326380968094, 0.036871299147605896, 0.08315159380435944, -0.01640699803829193, -0.020085880532860756, 0.016156015917658806, 0.038354095071554184, -0.007987958379089832, 0.017280805855989456, -0.0051147472113370895, 0.00648912787437439, -0.05800635740160942, 1.145291025750339e-05, -0.04248359799385071, -0.060891084372997284, -0.05684792995452881, -0.015501926653087139, 0.013350444845855236, -0.0006507885409519076, 0.019676847383379936, 0.015402265824377537, 0.052429161965847015, 0.001968993805348873, 0.01041609700769186, 0.01767989806830883, -0.021006150171160698, -0.058485839515924454, -0.06889361888170242, 0.0639859214425087, 0.029975958168506622, 0.022808127105236053, 0.031448908150196075, 0.07340440899133682, 0.02729358896613121, 0.05395696312189102, -0.00581020163372159, -0.022612635046243668, -0.00536998501047492, 0.012343580834567547, -0.005887121427804232, -0.026688039302825928, -0.015030638314783573, 0.012696831487119198, 0.04870394244790077, 0.012633314356207848, -0.018644077703356743, -0.07901187241077423, -0.08752722293138504, -0.0022897357121109962, 0.010770712047815323, 0.014505520462989807, -0.09416409581899643, 0.0004793071420863271, 0.03283032402396202, -0.04289063438773155, -0.08179327845573425, -0.057793863117694855, -0.01297734398394823, 0.03760221228003502, -0.06459517031908035, 0.03360287472605705, -0.004925709683448076, -0.04334568604826927, -0.040750037878751755, -0.0158659890294075, -0.05344042927026749, -0.011583392508327961, 0.05357319861650467, -0.07537452131509781, -0.022441547363996506, 0.09350031614303589, -0.03946594521403313, -0.036014165729284286, 0.04984953626990318, 0.0008575847605243325, 0.1023760586977005, 0.0044963303953409195, 0.01998019404709339, 0.053785182535648346, -0.07606254518032074, -0.07588524371385574, -0.07145844399929047, -0.05005015805363655, -0.06697262078523636, 0.0022250008769333363, -0.05334629863500595, -0.017972515895962715, 0.09120537340641022, -0.003218841040506959, -0.04601076990365982, 0.05959134176373482, -0.10171661525964737, 0.03943397104740143, -0.059057239443063736, -0.023265480995178223, 0.02563915029168129, -0.003640143433585763, 0.033833879977464676, -0.009260205551981926, -0.06492368131875992, 0.03435962647199631, 0.04271141067147255, 0.010489484295248985, 0.019840538501739502, 0.07815833389759064, -0.08266661316156387, -0.02183658443391323, -0.03091638721525669, -0.008115941658616066, 0.029693178832530975, -0.07540948688983917, 0.05108870193362236, 0.03157737851142883, -0.0851101279258728, 0.04176175966858864, -0.045292533934116364, 0.018976040184497833, 0.04509126767516136, -0.03704773634672165, 0.0005875899805687368, 0.05234156548976898, 0.010710583068430424, -0.07870838046073914, -0.010853085666894913, 0.06536082178354263, -0.07121702283620834, 0.057079020887613297, 0.008643213659524918, 0.015100760385394096, 0.03439740464091301, -0.007606267463415861, -0.03689802810549736, 0.04393801465630531, 0.017618924379348755, 0.039625588804483414, -0.09918665885925293, -0.09229757636785507, -0.0953102856874466, -0.0010616953950375319, -0.0064203510992228985, 0.03742031380534172, -0.02211310714483261, 0.053533680737018585, -0.05578483268618584, -0.04196624085307121, 0.07403450459241867, 0.033655665814876556, 0.010891655460000038, -0.012737695127725601, 0.04775302857160568, 0.058246761560440063, 0.05858221277594566, 0.05826100707054138, 0.01941119320690632, 0.0010937860934063792, -0.02644473873078823, 0.062264587730169296, 0.045947495847940445, -0.02744109556078911, 0.030093111097812653, 0.07393885403871536, -0.044950153678655624, 0.07687284052371979, 0.04417770728468895, -0.03303435444831848, 0.03874652832746506, 0.06106305494904518, -0.014257911592721939, 0.028015686199069023, 0.0011176386615261436, 0.002668168628588319, -0.02510380931198597, -0.056809891015291214, -0.06793638318777084, -0.04435613006353378, -0.09510558098554611, 0.02593877539038658, 0.06097329780459404, -0.04196861758828163, 0.026428043842315674, 0.021190010011196136, 0.017211120575666428, 0.05277116596698761, -0.05143854767084122, 0.04395335540175438, -0.012260950170457363, -0.052783798426389694, -0.09779851883649826, 0.010838030837476254, -0.10099063813686371, -0.0019370531663298607, -0.014193673618137836, 0.03999270871281624, 0.027434438467025757, 0.08314962685108185, 0.041163764894008636, 0.06224123388528824, -0.018175894394516945, 0.0704997181892395, 0.0012865541502833366, 0.08571186661720276, -0.06927868723869324, 0.047142695635557175, -0.06800557672977448, 0.006930980831384659, -0.05218777433037758, 0.06829670071601868, 5.312880966812372e-05, 0.017596464604139328, -0.05623118206858635, 0.03400733694434166, -0.01803123764693737, 0.008882449939846992, 0.049834996461868286, 0.09268715232610703, 0.003120291978120804, -0.06376585364341736, -0.05438048020005226, -0.05876128003001213, -0.07135279476642609, -0.011014287360012531, -0.10573668032884598, -0.059461724013090134, -0.0017550229094922543, 0.06444097310304642, 0.0429648831486702, -0.04523082077503204, -0.04186629876494408, 0.037968121469020844, -0.0774189755320549, -0.05798903852701187, -0.07658138871192932, 0.040370047092437744, 0.020437948405742645, -0.06376855820417404, 0.03781430423259735, 0.061908259987831116, -0.032817281782627106, 0.03399898111820221, -0.05585279315710068, 0.011775469407439232, -0.10090648382902145, -0.08940439671278, -0.07320675998926163, 0.08785917609930038, -0.04045521095395088, -0.09770762920379639, 0.02870400995016098, -0.003961327951401472, -0.04668501019477844, -0.009339935146272182, -0.022148584946990013, -0.0895923599600792, 0.03430038318037987, 0.04675920680165291, 0.024411577731370926, 0.09935447573661804, 0.07848041504621506, -0.007018359377980232, 0.05530335381627083, 0.06460268795490265, -0.06853355467319489, -0.07579712569713593, -0.07943391799926758, 0.05571074038743973, -0.08170564472675323, 0.018566736951470375, 0.06210806965827942, 0.039783500134944916, -0.010012583807110786, 0.03457175940275192, -0.05410460755228996, -0.04035894572734833, -0.00942157581448555, -0.011174940504133701, 0.08358265459537506, 0.042482055723667145, -0.009062448516488075, 0.021090982481837273, -0.053349096328020096, -0.021957602351903915, -0.02022821456193924, -0.02745581418275833, 0.01567699946463108, 0.009093278087675571, -0.07306385785341263, 0.0046450369991362095, -0.013490760698914528, -0.0989697203040123, -0.011989729478955269, 0.0543806292116642, 0.05623224005103111, -0.042621396481990814, 0.0757167711853981, 0.05214768648147583, 0.060156840831041336, 0.041199419647455215, 0.03165553882718086, -0.032669901847839355, -0.04673398286104202, -0.00044754211558029056, -0.07744937390089035, 0.04550006613135338, -0.04816536605358124, -0.03474253788590431, 0.047227565199136734, 0.09280180186033249, -0.0770980566740036, 0.030619153752923012, 0.021115537732839584, -0.005044844467192888, 0.03836296871304512, 0.007453444879502058, -0.01809227652847767, -0.09071876853704453, 0.020763332024216652, -0.07732319086790085, -0.002768290229141712, -0.03497260808944702, -0.034706369042396545, -0.002078449819236994, -0.08658986538648605, -0.02814151719212532, -0.0106388283893466, 0.07256090641021729, 0.04406212270259857, -0.0478433296084404, 0.022158466279506683, -0.08692250400781631, -0.009548462927341461, 0.013500050641596317, -0.0606696680188179, -0.02415507286787033, -0.023305336013436317, -0.03510894998908043, -0.07785344123840332, -0.05594656243920326, -0.0760083869099617, -0.07556583732366562, -0.08076921850442886, 0.05340971797704697, 0.017723875120282173, 0.050121892243623734, 0.045331284403800964, -0.019702479243278503, -0.02576732449233532, 0.018892310559749603, -0.004808517172932625, -0.03728894516825676, -0.08014883100986481, -0.09870732575654984, -0.04171020910143852, -0.056055642664432526, -0.11172349005937576, -0.00659632682800293, 0.06072671711444855, -0.041068851947784424, -0.010163368657231331, -0.01590484380722046, -0.04593614116311073, 0.09264053404331207, 0.005205899477005005, -0.03741852194070816, -0.06588800251483917, 0.045663099735975266, -0.05008335039019585, 0.05931563675403595, -0.04014255478978157, -0.0012871874496340752, 0.06251691281795502, 0.02276247926056385, -0.0033724233508110046, 0.019875258207321167, -0.02149055153131485, 0.049463387578725815, 0.09423043578863144, -0.09713667631149292, -0.02725306525826454, -0.032117411494255066, 0.10535431653261185, -0.08340822905302048, -0.02814481221139431, -0.04520874097943306, 0.027939656749367714, 0.1012863740324974, 0.01733935996890068, -0.08673332631587982, -0.050321463495492935, -0.06364470720291138, 0.030034974217414856, -0.049890317022800446, -0.07426469773054123, -0.05586676299571991, -0.0534035749733448, 0.05147282034158707, 0.01572970300912857, 0.048886738717556, 0.01806529611349106, -0.0035189965274184942, 0.037475716322660446, -0.07797598093748093, 0.032001957297325134, 0.03598156198859215, 0.08138248324394226, 0.0640813335776329, 0.06384093314409256, -0.005048898980021477, 0.06015186011791229, -0.014664145186543465, -0.07892172783613205, -0.0721278116106987, 0.06312484294176102, -0.0458064042031765, 0.04306964576244354, -0.09706699848175049, -0.0336838960647583, -0.0749346986413002, 0.045874275267124176, -0.08173248916864395, -0.0528738833963871, 0.0409524105489254, 0.07394279539585114, 0.0028612399473786354, 0.0957331657409668, 0.020385727286338806, -0.026471570134162903, -0.042766083031892776, 0.017650056630373, 0.008583015762269497, 0.03341355919837952, -0.14114338159561157, -0.013410940766334534, -0.04643743857741356, 0.021380087360739708, -0.06030239164829254, -0.08829259872436523, -0.1013823002576828, 0.08825637400150299, -0.004626706708222628, -0.09532178938388824, -0.01935753785073757, 0.022792303934693336, 0.04273500666022301, -0.11149192601442337, -0.08918517082929611, -0.11268249154090881, -0.03604898974299431, 0.01610557548701763, 0.05517807975411415, 0.005618041846901178, 0.09847066551446915, -0.07482240349054337, 0.06817816942930222, -0.02617076225578785, 0.0647139772772789, -0.09832233935594559, -0.043541762977838516, 0.01885192282497883, -0.03194485604763031, 0.0352865569293499, 0.04908892884850502, -0.08853861689567566, 0.04999402165412903, -0.012260165996849537, 0.009399781003594398, -0.06019272282719612, -0.014180085621774197, -0.01298437174409628, 0.06553778797388077, 0.04709544777870178, -0.05724778771400452, -0.09945216029882431, -0.053428441286087036, 0.029162148013710976, -0.05261487513780594, -0.033581748604774475, -0.043950408697128296, -0.10603884607553482, -0.10332994163036346, 0.05850421264767647, -0.017782993614673615, 0.011918537318706512, 0.030952833592891693, 0.029151076450943947, -0.05174625292420387, -0.09562550485134125, 0.05201375111937523, -0.07465886324644089, -0.0168538186699152, 0.01508189458400011, 0.0641918033361435, -0.03676486015319824, -0.0739208459854126, -0.025416014716029167, -0.059753838926553726, 0.050193123519420624, -0.02349727414548397, -0.005414072889834642, -0.0068233562633395195, 0.009534290991723537, 0.049398425966501236, -0.11697498708963394, -0.08187806606292725, 0.08291161805391312, -0.09748918563127518, -0.06740138679742813, -0.00031813106033951044, -0.010312395170331001, 0.023796163499355316, -0.022848479449748993, -0.02255067601799965, -0.10048260539770126, 0.1021883562207222, -0.06650861352682114, -0.025376969948410988, 0.061207108199596405, 0.05628261715173721, 0.0621349923312664, 0.030215585604310036, -0.04305926337838173, 0.03480115160346031, 0.03790952265262604, -0.06239777058362961, -0.07698644697666168, 0.04572907090187073, 0.06887279450893402, -0.06634538620710373, -0.10757352411746979, -0.052996911108493805, 0.017575424164533615, 0.0008223616168834269, -0.04810598865151405, -0.05352858453989029, -0.10032089799642563, -0.013178186491131783, 0.02600063756108284, -0.005311650224030018, 0.03044530563056469, 0.01792054995894432, 0.06734026223421097, 0.04206659644842148, 0.07183973491191864, 0.023503290489315987, -0.12183897197246552, -0.0457054041326046, 0.058301135897636414, -0.030773157253861427, 0.04478193819522858, -0.06823141127824783, -0.009160485118627548, 0.07185622304677963, 0.007452371995896101, 0.08219479769468307, -0.07646171003580093, -0.09878620505332947, 0.07220904529094696, -0.015586496330797672, -0.02045353502035141, -0.017596270889043808, 0.014667732641100883, 0.029185093939304352, 0.0609644278883934, -0.08815934509038925, -0.080230712890625, -0.07795565575361252, -0.06010985001921654, 0.03960411623120308, 0.011226196773350239, -0.02992139756679535, -0.029195014387369156, -0.09740780293941498, -0.03382548317313194, 0.01757955364882946, 0.0741652101278305, 0.0045358529314398766, -0.005003529600799084, 0.024331530556082726, 0.028254013508558273, -0.0019228497985750437, -0.025569504126906395, -0.01709263026714325, 0.06194320693612099, -0.057407185435295105, 0.00579607579857111, -0.06394285708665848, -0.07127615809440613, -0.03240792453289032, -0.018969783559441566, 0.027167286723852158, 0.07274509221315384, 0.10495244711637497, 0.04062877595424652, -0.08200222998857498, -0.041542571038007736, -0.04289599508047104, 0.0027487557381391525, 0.057718273252248764, 0.020621832460165024, 0.0488249734044075, 0.0779481753706932, 0.02146797813475132, 0.05867907032370567, -0.0065852017141878605, -0.004588230513036251, 0.048379428684711456, -0.10048150271177292, -0.0167101938277483, -0.08082552254199982, 0.07754071056842804, 0.004372492432594299, -0.07613034546375275, 0.011277114041149616, 0.0217391699552536, -0.07987037301063538, -0.013645174913108349, 0.042616117745637894, 0.011720801703631878, 0.08755769580602646, -0.04889125004410744, -0.05104715749621391, -0.02872229553759098, 0.059496957808732986, 0.06438808888196945, -0.02231081947684288, 0.04152815043926239, -0.061326924711465836, 0.03016502968966961, -0.040262673050165176, 0.05853487551212311, 0.08295943588018417, -0.10181909799575806, -0.048297882080078125, 0.07429742813110352, 0.028399180620908737, 0.03224160894751549, -0.09472786635160446, 0.06445696949958801, 0.031083200126886368, -0.048653170466423035, -0.014937704429030418, -0.08680287003517151, 0.0775911808013916, -0.08316542208194733, -0.020678313449025154, 0.043517425656318665, 0.025724725797772408, 0.007525935769081116, -0.015634410083293915, -0.011369551531970501, -0.00015975265705492347, -0.011059450916945934, -0.03577949479222298, 0.07556185871362686, 0.06697668880224228, 0.0683138519525528, 0.051656950265169144, -0.11835815757513046, 0.053459566086530685, -0.023571746423840523, -0.04829896613955498, -0.049089524894952774, 0.015045666135847569, -0.07743595540523529, -0.02781429886817932, -0.06701291352510452, 0.07818615436553955, -0.06039309501647949, -0.03412097692489624, -0.058525118976831436, -0.08268987387418747, 0.07153292745351791, -0.005205388180911541, -0.02771466039121151, -0.036044053733348846, -0.12061545997858047, 0.07780393958091736, 0.05115782842040062, -0.01598055474460125, 0.08071208000183105, 0.0801839604973793, -0.007262386381626129, -0.07465779036283493, 0.0029203896410763264, 0.008825836703181267, 0.0914001539349556, -0.06916845589876175, 0.05854243412613869, 0.019726276397705078, 0.06344929337501526, 0.08454607427120209, -0.06749506294727325, -0.07506686449050903, 0.012624727562069893, -0.07252578437328339, 0.08465157449245453, 0.006356050726026297, -0.016357412561774254, -0.039814259856939316, -0.033027537167072296, -0.06735484302043915, -0.022471070289611816, -0.07439228147268295, 0.0890403762459755, 0.038591235876083374, 0.015218584798276424, 0.007742659654468298, -0.03684525191783905, -0.049585141241550446, -0.04668102040886879, -0.005970929749310017, 0.013213688507676125, 0.05988416448235512, -0.029988856986165047, -0.027911337092518806, -0.00943268183618784, -0.02852986752986908, 0.04121832549571991, -0.013872169889509678, 0.08460087329149246, 0.0548948310315609, -0.006881197914481163, -0.019920872524380684, -0.029201870784163475, -0.07063860446214676, 0.08819857984781265, -0.02031691186130047, -0.0053259264677762985, -0.03605160117149353, 0.08745793998241425, 0.0007556655909866095, 0.07502520084381104, -0.043254539370536804, 0.0020272238180041313, -0.05891506001353264, -0.03330421820282936, -0.006869738921523094, -0.0008758747135289013, 0.008315706625580788, -0.008467540144920349, 0.0980471596121788, 0.05047762393951416, 0.07570867985486984, 0.05425197258591652, 0.047355424612760544, -0.0734010711312294, -0.09798117727041245, 0.0015362275298684835, 0.08159543573856354, -0.08412758260965347, -0.03345440328121185, -0.041208986192941666, 0.01958751678466797, -0.1012527346611023, -0.01690608076751232, 0.022699426859617233, 0.08011318743228912, 0.016972528770565987, -0.019455567002296448, -0.04258731007575989, -0.0090302349999547, 0.027224689722061157, -0.0484018437564373, -0.05809401348233223, 0.022918522357940674, -0.0714133232831955, -0.05827224627137184, -0.0651223435997963, 0.007536280434578657, -0.02359018474817276, -0.09111401438713074, -0.02433978021144867, -0.03827379271388054, 0.028081540018320084, -0.05196915939450264, 0.08386386185884476, -0.07097966969013214, -0.08438967913389206, 0.06463269889354706, -0.048526592552661896, 0.08003757148981094, 0.051224857568740845, 0.01953371986746788, 0.05136609077453613, 0.045604217797517776, -0.06920213997364044, -0.057393789291381836, -0.08866758644580841, 0.019922448322176933, -0.050738587975502014, 0.03969023376703262, -0.024938881397247314, 0.09439479559659958, -0.09480054676532745, 0.03468615934252739, 0.012968660332262516, -0.026190944015979767, -0.08494819700717926, -0.05036341771483421, 0.0374666228890419, 0.00044276067637838423, -0.04907616227865219, 0.02762915939092636, -0.05197141692042351, 0.04536277800798416, 0.04347237944602966, 0.033079151064157486, 0.027772044762969017, -0.05492575839161873, 0.016422487795352936, -0.002385178115218878, -0.04304340481758118, 0.060569025576114655, 0.013019915670156479, 0.005724439863115549, 0.07515274733304977, -0.025354033336043358, -0.027935737743973732, 0.056736622005701065, -0.006940243300050497, -0.09193267673254013, -0.027701180428266525, -0.027072738856077194, 0.026487987488508224, 0.08250334858894348, 0.031255509704351425, 0.05820494890213013, 0.013080338016152382, 0.010490641929209232, 0.042746402323246, 0.03746260702610016, -0.04989595711231232, -0.0463387705385685, -0.019426142796874046, -0.0063830167055130005, -0.012986317276954651, -0.014800249598920345, 0.04692696034908295, -0.0591118149459362, -0.06542108207941055, -0.011189735494554043, -0.04801938682794571, -0.0693650096654892, -0.06586132198572159, 0.03418868035078049, -0.031770333647727966, -0.09196127951145172, 0.006004123017191887, 0.03749064728617668, -0.03653637692332268, 0.013715418055653572, 0.0010696176905184984, -0.012874896638095379, -0.06393132358789444, -0.03179894760251045, -0.03753788024187088, 0.06999023258686066, 0.0584220252931118, 0.0680420771241188, -0.04994823411107063, 0.055220864713191986, -0.06551395356655121, -0.07562781870365143, 0.04465232789516449, 0.03496956825256348, -0.04463031142950058, -0.09132099896669388, -0.04288649186491966, 0.022679220885038376, 0.09808539599180222, 0.04882018268108368, -0.06518290936946869, -0.06694407016038895, -0.049545250833034515, 0.06619229912757874, 0.01870126463472843, -0.023313624784350395, 0.025050323456525803, -0.06066897138953209, -0.04999110847711563, 0.057073090225458145, 0.07364989817142487, 0.001484496402554214, -0.06422439217567444, 0.07083814591169357, -0.006165654864162207, -0.006707330234348774, 0.026989011093974113, -0.001443703193217516, 0.05013656243681908, 0.029583441093564034, 0.08167373389005661, 0.08163567632436752, 0.06390578299760818, -0.045654576271772385, -0.032546479254961014, -0.05862830951809883, 0.05597677454352379, -0.06625480204820633, 0.04897478222846985, -0.027253692969679832, 0.055252693593502045, -0.04245402663946152, -0.09131352603435516, 0.0814753845334053, 0.0037663732655346394, -0.050374895334243774, 0.05706057697534561, -0.015314016491174698, 0.07192985713481903, 0.043957166373729706, 0.0018480040598660707, 0.052722182124853134, 0.053039394319057465, 0.00799543783068657, 0.061753544956445694, 0.05960731580853462, 0.006612333003431559, -0.0002669768291525543, 0.0038239762652665377, 0.030863557010889053, 0.06082761660218239, 0.0030200378969311714, -0.0161703210324049, -0.02378081902861595, 0.03336712718009949, 0.07032221555709839, 0.03635275736451149, 0.0003413477388676256, 0.07637935131788254, 0.016333360224962234, -0.004075373522937298, -0.05545930564403534, 0.02964768372476101, -0.0018969771917909384, 0.06623750180006027, -0.035377513617277145, 0.041960857808589935, 0.03599904105067253, 0.01576591469347477, 0.09292258322238922, 0.026073362678289413, 0.03014691360294819, -0.0391814298927784, 0.02607148513197899, -0.028026802465319633, 0.05753854289650917, 0.05994274467229843, -0.013203720562160015, 0.021617108955979347, -0.02797706238925457, -0.0043331594206392765, 0.05021842569112778, 0.09048135578632355, 0.08243588358163834, -0.006951617076992989, -0.02568214386701584, 0.035063572227954865, -0.0563105046749115, 0.06539946049451828, 0.0441226102411747, 0.08084484934806824, 0.015644362196326256, 0.10310178995132446, -0.026322918012738228, -0.028405480086803436, -0.04711485281586647, 0.0877302885055542, -0.08037474006414413, 0.038901690393686295, -0.07734787464141846, -0.0435384064912796, -0.07071315497159958, -0.07848690450191498, -0.046296458691358566, 0.0386992022395134, 0.008494319394230843, -0.08899262547492981, -0.02174755558371544, 0.06166940554976463, 0.05021514371037483, -0.029171107336878777, 0.06118679791688919, -0.04053362086415291, 0.08168832957744598, 0.0034454341512173414, 0.09961532801389694, -0.034342024475336075, -0.08183673024177551, -0.016169201582670212, -0.0003634098684415221, 0.0648856833577156, -0.061767447739839554, -0.06543070822954178, -0.02065015584230423, -0.052344392985105515, 0.07012376189231873, 0.07000884413719177, -0.036862701177597046, 0.031628165394067764, 0.05798209086060524, -0.023251837119460106, 0.03248414769768715, -0.08883894234895706, -0.037308964878320694, -0.058361828327178955, -0.004508917685598135, 0.04378712177276611, 0.030900048092007637, 0.05281178280711174, -0.07754272222518921, -0.0798858031630516, -0.08468366414308548, -0.02905251830816269, -0.06450584530830383, 0.04877161234617233, 0.09968026727437973, -0.08426432311534882, -0.07626233994960785, 0.032574329525232315, 0.04009680822491646, -0.0428781695663929, -0.04036600887775421, 0.01690703257918358, -0.07667916268110275, 0.09220164269208908, 0.10622154176235199, -0.02671508491039276, 0.04941219091415405, -0.05935141444206238, 0.07634427398443222, -0.032162051647901535, -0.0020973586943000555, 0.034524690359830856, 0.02735854871571064, -0.02832845225930214, -0.021464873105287552, 0.07598613947629929, 0.020002126693725586, 0.004266523290425539, -0.041012465953826904, 0.03740560635924339, -0.05254944786429405, -0.05682608112692833, 0.020276134833693504, -0.0036512953229248524, 0.05472930893301964, 0.0192323736846447, -0.09235424548387527, -0.09208213537931442, 0.0361238569021225, -0.015595217235386372, 0.010852303355932236, 0.010120294988155365, 0.016035884618759155, 0.02661203220486641, 0.010222118347883224, -0.04740710183978081, 0.029037363827228546, 0.05055082589387894, -0.09195072948932648, -0.044949933886528015, 0.04043140262365341, 0.03598588705062866, 0.04255346581339836, -0.04000461474061012, 0.0014209719374775887, -0.01970910094678402, -0.01626216620206833, 0.10689758509397507, -0.034569986164569855, -0.08148787915706635, 0.07617583125829697, -0.0057424395345151424, 0.02394017018377781, 0.023497669026255608, 0.0805988684296608, -0.014806891791522503, -0.0451485738158226, 0.07202380150556564, -0.03792644292116165, 0.08875501900911331, -0.017052121460437775, -0.060750819742679596, 0.05598175525665283, -0.057350147515535355, -0.005103087518364191, -0.030982090160250664, 0.027805892750620842, -0.041442517191171646, 0.08955394476652145, -0.025699522346258163, 0.07865455746650696, 0.049875274300575256, 0.01100010797381401, 0.009694104082882404, 0.056328777223825455, 0.0967009887099266, 0.0725146234035492, -0.11335856467485428, -0.048492591828107834, 0.03173939138650894, -0.0011128506157547235, 0.05701916292309761, -0.052028633654117584, -0.0910181924700737, 0.04534611105918884, 0.06832876801490784, -0.05146147683262825, 0.04455307126045227, -0.03137528523802757, 0.02252654731273651, -0.0617901012301445, -0.053727101534605026, -0.05904470384120941, 0.023494010791182518, -0.0037089954130351543, -0.024124156683683395, -0.004370815120637417, -0.00813452061265707, 0.07613972574472427, -0.060544975101947784, -0.029709096997976303, -0.017503442242741585, -0.010547708719968796, -0.07091419398784637, -0.07014232873916626, 0.02632509358227253, -0.04974735900759697, 0.13105632364749908, -0.031381916254758835, -0.023361239582300186, 0.05334954336285591, -0.06170836463570595, 0.09068165719509125, 0.030132632702589035, -0.07261904329061508, 0.06048279628157616, 0.08254193514585495, 0.0060865916311740875, -0.05563311651349068, 0.014171553775668144, 0.039179783314466476, -0.0854257196187973, -0.02540593594312668, -0.027995290234684944, -0.0017600997816771269, 0.04948839172720909, 0.0687250941991806, -0.02177710272371769, -0.011421640403568745, 0.06855694204568863, 0.002164756879210472, -0.0743083655834198, -0.022142846137285233, 0.024638429284095764, 0.1021861657500267, -0.021571161225438118, 0.0038392869755625725, -0.0825095847249031, 0.07273445278406143, -0.05324157699942589, -0.057314857840538025, 0.06628552824258804, -0.043038249015808105, -0.041493505239486694, -0.03446276858448982, -0.02395278960466385, 0.06763105094432831, -0.06662445515394211, 0.025030074641108513, 0.010615749284625053, 0.09042602777481079, 0.06203272566199303, -0.04598527401685715, -0.02993413433432579, 0.11388998478651047, -0.06177322939038277, 0.017907431349158287, -0.06747917830944061, -0.06209283322095871, 0.004256721120327711, 0.03224288672208786, 0.06266862899065018, -0.08389817178249359, 0.03322219103574753, 0.033133603632450104, -0.027580764144659042, -0.01640775054693222, 0.028091944754123688, 0.004348744638264179, 0.028724215924739838, -0.0056182318367064, 0.04926733300089836, -0.0661156177520752, 0.07593711465597153, 0.04050350934267044, 0.04037567973136902, 0.06638132780790329, -0.04994972050189972, -0.004444393329322338, 0.07924874871969223, 0.07143265008926392, -0.07852741330862045, 0.04545348510146141, -0.026787618175148964, 0.08138024061918259, -0.039888184517621994, -0.009119970723986626, -0.024855472147464752, -0.00020624470198526978, -0.021778786554932594, 0.03786469250917435, 0.015502440743148327, 0.006283109076321125, 0.07795826345682144, 0.02330353483557701, 0.062259286642074585, -0.05820983648300171, 0.06199103966355324, -0.028535375371575356, -0.01827782206237316, -0.026239298284053802, 0.02349741943180561, -0.07848816365003586, -0.0016402642941102386, 0.02918240986764431, 0.06461715698242188, -0.06327017396688461, 0.07708846777677536, 0.08590201288461685, 0.10574547946453094, 0.03236177936196327, 0.08091993629932404, -0.0369066558778286, -0.02727007307112217, -0.09528446197509766, -0.08308034390211105, -0.07821188122034073, 0.05223821476101875, -0.043278828263282776, -0.0008730394183658063, -0.04172411188483238, 0.0401296466588974, -0.04544553905725479, -0.06668730825185776, -0.013580107130110264, 0.03721311315894127, 0.010239284485578537, -0.06494960188865662, -0.018344948068261147, -0.0702282041311264, -0.0822208970785141, 0.06113751605153084, -0.04905714839696884, -0.02465122379362583, 0.01322009414434433, 0.0938354879617691, -0.05321067199110985, -0.01454420667141676, -0.09679088741540909, -0.09055524319410324, 0.01085707824677229, -0.018094968050718307, 0.06871740520000458, -0.011662919074296951, 0.06489735841751099, -0.04424356296658516, 0.029881903901696205, -0.03184065595269203, -0.08546529710292816, -0.03278843313455582, -0.005333220586180687, 0.005175989121198654, 0.06146005168557167, 0.012394482269883156, -0.07086948305368423, 0.002754546469077468, -0.05909465625882149, 0.02266874350607395, -0.0487980879843235, 0.06152540445327759, 0.04075281322002411, 0.03802340477705002, -0.0447763130068779, 0.03901831433176994, 0.007493302691727877, 0.07659459859132767, -0.03665293753147125, -0.03298306465148926, 0.013202629052102566, -0.011716767214238644, 0.0478079691529274, 0.027659695595502853, -0.07829385995864868, -0.039495501667261124, -0.053697481751441956, -0.030858220532536507, 0.04267817735671997, 0.07049522548913956, 0.031956110149621964, 0.056191496551036835, 0.07078201323747635, 0.03917159512639046, -0.020982231944799423, -0.04019009321928024, -0.016283690929412842, -0.028080854564905167, -0.03612358123064041, 0.014104487374424934, 0.02659018337726593, -0.0177166685461998, -0.07168647646903992, 0.041853394359350204, 0.02293149195611477, 0.04698704183101654, 0.03329721465706825, -0.0753156989812851, 0.07895741611719131, 0.07730657607316971, 0.061900459229946136, 0.033586930483579636, 0.06601357460021973, -0.07244855910539627, 0.00714823417365551, -0.03790827468037605, 0.05570395290851593, 0.01228222157806158, -0.04285769164562225, 0.05536403879523277, 0.019377121701836586, 0.08601551502943039, 0.051022689789533615, -0.011929073370993137, 0.07268745452165604, 0.10173407196998596, -0.018652357161045074, 0.021636443212628365, -0.06457354873418808, 0.04709718003869057, -0.009787391871213913, -0.059265367686748505, -0.03347790986299515, -0.015037442557513714, 0.03664691746234894, -0.008305929601192474, -0.06940854340791702, 0.013463198207318783, -0.08523373305797577, 0.0553208589553833, 0.06581436842679977, 0.030748611316084862, 0.015718691051006317, 0.01342475600540638, -0.046309132128953934, 0.02361433021724224, -0.0582461841404438, 0.04583458974957466, -0.010195243172347546, 0.0008174505783244967, 0.0773257240653038, -0.02171538583934307, 0.02274804748594761, -0.032999731600284576, -0.04308704286813736, -0.09948386996984482, -0.12448727339506149, 0.07898493111133575, -0.08790674060583115, -0.015198265202343464, -0.0659186840057373, 0.049463361501693726, 0.09557851403951645, -0.08491726219654083, 0.08227505534887314, 0.009408394806087017, 0.0419873408973217, -0.08120502531528473, 0.024879777804017067, -0.03852910175919533, -0.03169497847557068, 0.004323161207139492, -0.04457908123731613, 0.07340016961097717, -0.08870870620012283, -0.017511513084173203, 0.01584586314857006, 0.08020460605621338, -0.06711979955434799, -0.0350433886051178, 0.002581281354650855, -0.08490812033414841, -0.0017977682873606682, -0.10141152143478394, 0.018417811021208763, 0.0489838607609272, 0.0026025341358035803, 0.03245603293180466, 0.06226058304309845, 0.06310350447893143, 0.0330062210559845, 0.08940351009368896, 0.025658482685685158, -0.05607745796442032, 0.005950598046183586, -0.038184721022844315, -0.0596373975276947, -0.06974110007286072, -0.026653308421373367, -0.032438091933727264, 0.06599321216344833, 0.06240407004952431, -0.012994932010769844, 0.0014525294536724687, -0.015356522053480148, 8.591698133386672e-05, 0.06270845234394073, 0.06832569092512131, 0.03067205287516117, -0.00846666656434536, -0.09092438220977783, 0.05549925938248634, -0.09706077724695206, 0.09628831595182419, -0.014337295666337013, -0.04737896844744682, 0.023008283227682114, 0.07693033665418625, -0.06415721029043198, 0.02055463008582592, -0.029495585709810257, -0.0032839763443917036, -0.02160479687154293, 0.03856019675731659, -0.01728946529328823, -0.007402379065752029, -0.02861606702208519, -0.03038652427494526, -0.014909329824149609, -0.10207866877317429, -0.03468777984380722, -0.07259481400251389, -0.07244189828634262, -0.0023416930343955755, -0.0825904831290245, -0.015571240335702896, 0.055642783641815186, 0.07818998396396637, 0.02794354408979416, -0.09355942159891129, -0.04681773483753204, 0.060210805386304855, -0.053387805819511414, 0.02202373929321766, -0.025720132514834404, -0.09921630471944809, -0.0989285260438919, -0.01019222941249609, -0.055687543004751205, 0.09846161305904388, 0.0027026173193007708, 0.028288189321756363, 0.06766121834516525, -0.055074870586395264, -0.054281070828437805, -0.0397610105574131, -0.04519308730959892, -0.07423465698957443, -0.04667722433805466, -0.05998681113123894, -0.023045748472213745, -0.006608983501791954, -0.10768653452396393, -0.07856404036283493, 0.04924686998128891, -0.04070168733596802, -0.0326555110514164, 0.010620856657624245, 0.023970818147063255, 0.068936288356781, -0.005961754824966192, 0.08267631381750107, -0.01609639637172222, -0.010685578919947147, -0.09861747920513153, 0.024205660447478294, -0.03046558052301407, -0.004487974569201469, 0.022847600281238556, -0.09649217873811722, 0.003914709202945232, -0.06708057224750519, -0.014389379881322384, 0.033117569983005524, -0.06851950287818909, -0.006562615744769573, -0.0668068677186966, -0.019616544246673584, 0.08257854729890823, 0.07476212084293365, -0.06234382465481758, -0.01773979142308235, 0.018996676430106163, 0.03470601513981819, -0.09728821367025375, 0.01770055666565895, -0.054640136659145355, -0.036816179752349854, 0.008676710538566113, -0.06956164538860321, -0.029053451493382454, -0.09256893396377563, 0.07022755593061447, -0.040497131645679474, -0.028377564623951912, -0.008933588862419128, -0.10313114523887634, -0.05168360844254494, 0.10703162103891373, 0.024181660264730453, 0.010872474871575832, -0.033916860818862915, -0.0315677784383297, 0.0342266671359539, 0.012939047068357468, -0.030005525797605515, -0.06660625338554382, 0.04479415714740753, 0.016987042501568794, -0.07669859379529953, -0.07681874930858612, 0.01000313088297844, 0.09946077316999435, 0.05735129490494728, 0.04897458478808403, 0.032232556492090225, -0.05783109739422798, -0.04239972308278084, 0.0030915073584765196, 0.0011261749314144254, 0.0245809368789196, -0.01733858324587345, 0.06029848754405975, 0.015765052288770676, 0.0021483413875102997, 0.017165930941700935, 0.060544490814208984, -0.040093790739774704, -0.09101134538650513, -0.07175450772047043, -0.011163046583533287, 0.022478945553302765, 0.05761370062828064, -0.009223619475960732, 0.09385672211647034, -0.03129581734538078, -0.006438492797315121, -0.028999563306570053, -0.0010511789005249739, 0.0664844959974289, 0.01482907310128212, -0.05354338139295578, -0.014471604488790035, -0.05911613255739212, -0.05124276876449585, -0.020631153136491776, 0.028610819950699806, -0.03796590492129326, 0.01698251999914646, -0.0481470450758934, -0.029039911925792694, -0.028724126517772675, -0.013638429343700409, 0.048153530806303024, 0.05476389452815056, -0.00434104586020112, -0.05883863940834999, 0.0420871302485466, 0.0016588824801146984, -0.04401287063956261, 0.04366716742515564, 0.06256094574928284, 0.07216427475214005, -0.09880396723747253, 0.05320107564330101, 0.025484982877969742, 0.03960712254047394, 0.0698537677526474, -0.059265363961458206, -0.0818539410829544, -0.032891616225242615, -0.026997940614819527, 0.0015549793606624007, -0.04671834409236908, 0.009058349765837193, 0.027767781168222427, 0.0516623817384243, 0.0016772578237578273, 0.04288773983716965, -0.04503672942519188, 0.026803219690918922, 0.03505244106054306, 0.024038055911660194, -0.04762403666973114, -0.06256260722875595, -0.044498372822999954, 0.040277451276779175, 0.07109320163726807, -0.01856708712875843, -0.06073710322380066, -0.06963367760181427, -0.03011149913072586, 0.05163690075278282, -0.007929001934826374, -0.05024053528904915, 0.07997913658618927, 0.041818127036094666, -0.0010217712260782719, -0.030051909387111664, -0.052863556891679764, -0.08920592069625854, 0.050204887986183167, -0.056493259966373444, -0.04345151036977768, -0.05814855173230171, 0.028184643015265465, -0.07734458148479462, -0.05285828933119774, 0.05918997526168823, 0.033084966242313385, 0.039422184228897095, -0.11081503331661224, 0.011916648596525192, 0.03562299534678459, 0.06382928788661957, -0.07149838656187057, 0.018526893109083176, 0.03528543561697006, 0.059357933700084686, 0.07490632683038712, -0.03905334323644638, 0.047109268605709076, 0.019659394398331642, -0.013185400515794754, -0.01812960021197796, -0.009316070936620235, 0.033395376056432724, -0.012940628454089165, -0.048541367053985596, -0.09454169869422913, 0.04088926687836647, -0.10084889829158783, -0.06805926561355591, 0.08073204755783081, 0.025295352563261986, 0.023379936814308167, -0.010996894910931587, -0.07372049242258072, -0.08725259453058243, -0.043240975588560104, -0.08738178759813309, -0.042334575206041336, 0.05723782628774643, 0.09052787721157074, -0.09245206415653229, -0.04977975785732269, 0.005364832002669573, -0.06280517578125, -0.05950363725423813, -0.011904965154826641, 0.008492597378790379, 0.051647260785102844, -0.1101863831281662, 0.06525932997465134, -0.0037579145282506943, 0.011899537406861782, -0.06378009915351868, 0.016976313665509224, -0.039460040628910065, -0.01695328764617443, -0.06102462857961655, -0.06343110650777817, -0.03440432995557785, 0.08038119226694107, 0.08127444982528687, -0.03205425664782524, 0.004856116138398647, 0.02215646393597126, 0.03334246203303337, 0.04323577880859375, -0.03340637683868408, 0.042276669293642044, -0.027878059074282646, -0.038248274475336075, 0.04472209885716438, 0.024203483015298843, -0.07761672884225845, -0.08113221079111099, -0.06009376794099808, -0.010357162915170193, -0.04249979183077812, -0.07165250182151794, 0.051758456975221634, 0.020711280405521393, -0.010711431503295898, -0.05597102642059326, 0.07287783175706863, 0.005179941188544035, -0.04965856298804283, -0.09764789789915085, 0.06676962971687317, 0.00792615208774805, -0.024654459208250046, -0.076899453997612, -0.01814166083931923, -0.053140949457883835, -0.02156558819115162, 0.031637515872716904, -0.028852321207523346, 0.09091257303953171, -0.08746854215860367, 0.06990759819746017, -0.006555768661201, -0.006255913991481066, 0.002760512288659811, 0.0027088667266070843, -0.021410826593637466, -0.03732854500412941, 0.061335451900959015, -0.0321936160326004, 0.07318992912769318, 0.07373341917991638, 0.06537595391273499, -0.0283054132014513, 0.06733667850494385, 0.02189292199909687, -0.05107327178120613, 0.010392870754003525, -0.0044934991747140884, -0.03903460502624512, -0.04218418896198273, 0.12241695821285248, 0.00949802901595831, -0.028684571385383606, -0.07550116628408432, -0.009077209047973156, -0.07062841951847076, -0.032859545201063156, -0.024855658411979675, 0.022753754630684853, -0.09111723303794861, 0.0426480658352375, 0.03816237300634384, 0.04022349417209625, -0.046359170228242874, 0.020187273621559143, -0.056866180151700974, 0.03314697742462158, 0.04114418849349022, 0.0076205129735171795, -0.07564917951822281, -0.021029751747846603, -0.0400855578482151, 0.008124910295009613, -0.016406407579779625, -0.09185411781072617, 0.07187620550394058, 0.020277952775359154, -0.05946178361773491, 0.006016692146658897, 0.01097993366420269, -0.02925526723265648, 0.022187579423189163, -0.019528983160853386, 0.0312265083193779, -0.05669037625193596, 0.01487143523991108, -0.054702870547771454, 0.012802197597920895, 0.01995711587369442, 0.04354584589600563, 0.05596030503511429, 0.06436031311750412, -0.08817867934703827, 0.07378698140382767, -0.06463140994310379, 0.07386752963066101, 0.08344946056604385, -0.06911598145961761, 0.054206956177949905, -0.0354803204536438, 0.0563221201300621, -0.04867975786328316, -0.04079495742917061, 0.03549214079976082, -0.0289540346711874, -0.06679920107126236, -0.001604534569196403, 0.06187436729669571, -0.00010023433424066752, 0.05047856271266937, -0.026771532371640205, -0.014293545857071877, -0.07865100353956223, -0.07729539275169373, 0.035280779004096985, 0.06717623770236969, 0.05469907075166702, -0.007795881014317274, 0.03384263068437576, 0.05227337405085564, -0.024735083803534508, 0.004016256425529718, 0.08997062593698502, 0.05093551054596901, -0.08467515558004379, -0.08380910754203796, 0.017269663512706757, 0.008644926361739635, 0.0724552795290947, 0.023026658222079277, -0.11016631126403809, 0.061462465673685074, 0.04978511482477188, 0.06677192449569702, 0.06069048494100571, 0.05891190096735954, 0.07381011545658112, -0.06685860455036163, 0.08112965524196625, 0.0005256080767139792, 0.03996196389198303, 0.08709393441677094, 0.04419142007827759, 0.01845216192305088, -0.047166839241981506, 0.05897204577922821, -0.03301220387220383, -0.014604500494897366, 0.005826527252793312, -0.07187293469905853, 0.08986295759677887, -0.04333707317709923, 0.06101727485656738, -0.04700728505849838, -0.06330868601799011, -0.05620746314525604, 0.029014697298407555, -0.06984708458185196, 0.0249561108648777, 0.036239758133888245, -0.04314754158258438, -0.00020500959362834692, -0.05317779630422592, -0.009112686850130558, -0.11161161959171295, 0.039238691329956055, -0.024503901600837708, 0.07677134871482849, 0.07979835569858551, -0.0010183838894590735, 0.05151330307126045, 0.06375447660684586, -0.03768719360232353, -0.0004017410974483937, 0.043692056089639664, -0.022725030779838562, -0.08330042660236359, -0.005963490344583988, 0.05909479036927223, -0.002359552076086402, 0.08168978989124298, -0.10168749839067459, 0.06633149832487106, -0.0842648446559906, 0.049994587898254395, 0.05758053436875343, 0.03788571432232857, 0.06472732871770859, 0.02469347044825554, 0.0776478499174118, -0.036724500358104706, -0.050311751663684845, 0.04006762057542801, -0.07912758737802505, -0.07897686958312988, -0.04869190603494644, -0.01418988686054945, -0.004066816996783018, 0.04767593368887901, 0.030629128217697144, -0.009851173497736454, -0.018370451405644417, 0.046663109213113785, 0.021464983001351357, 0.005249265115708113, 0.03270181640982628, -9.230678551830351e-05, -0.04547697678208351, 0.00749250128865242, 0.06781782954931259, 0.08231347799301147, 0.09740087389945984, -0.0023925662972033024, -0.024491846561431885, 0.06686730682849884, -0.0028226233553141356, -0.10010948777198792, -0.0722164660692215, -0.07185588777065277, -0.05365573987364769, 0.027931535616517067, -0.027304714545607567, 0.04043179750442505, -0.012743216939270496, -0.01283975038677454, 0.031115831807255745, 0.07611498236656189, -0.04109013453125954, 0.09350623935461044, -0.04359493777155876, 0.0014859854709357023, -0.017931949347257614, -0.007417211774736643, -0.050863128155469894, -0.013578328303992748, 0.02888629026710987, 0.003817326854914427, 0.038626961410045624, -0.013900713995099068, 0.06388196349143982, -0.03532171994447708, -0.008664797060191631, -0.004277155734598637, -0.06600141525268555, 0.05602576211094856, 0.026594949886202812, -0.010182463563978672, 0.011436798609793186, -0.04667498543858528, -0.04318294674158096, -0.031487200409173965, -0.009591576643288136, -0.031514137983322144, 0.042517852038145065, 0.029656948521733284, 0.07778549939393997, 0.06876958161592484, 5.020821117796004e-05, -0.01348009705543518, -0.05206015333533287, 0.06403028219938278, -0.09421589970588684, -0.08748685568571091, -0.04990252107381821, 0.03097846731543541, 0.027361059561371803, -0.021926598623394966, 0.0515657402575016, -0.06593389064073563, 0.012265675701200962, -0.02687884494662285, -0.09061844646930695, 0.040692806243896484, -0.08514229208230972, 0.03903338685631752, -0.020083587616682053, -0.06991280615329742, 0.032926611602306366, 0.03984406217932701, 0.05565565824508667, 0.01581830531358719, -0.10212337970733643, -0.102122962474823, -0.03999613597989082, 0.007722928654402494, -0.02737255208194256, -0.04106258973479271, -0.018601128831505775, -0.07797085493803024, -0.08341843634843826, 0.027639323845505714, -0.06778117269277573, -0.05663038045167923, 0.041997332125902176, -0.022373024374246597, -0.07443363219499588, 0.03927259147167206, -0.0349857434630394, 0.029271801933646202, -0.09635543078184128, 0.020929964259266853, -0.05835048481822014, -0.00835702009499073, 0.028897225856781006, -0.013601581566035748, -0.05609237402677536, -0.004789430182427168, -0.0238399188965559, 0.058125339448451996, -0.011398139409720898, 0.0765211284160614, 0.011356404982507229, 0.010648583993315697, -0.08674245327711105, 0.03889448940753937, 0.0034135738387703896, -0.01503250002861023, -0.07258061319589615, -0.04609924927353859, -0.007996304892003536, 0.06424468755722046, -0.022923318669199944, -0.06739836186170578, 0.028584858402609825, 0.08039188385009766, 0.017626725137233734, -0.06925611943006516, 0.06668657064437866, -0.004546116106212139, 0.003700066590681672, -0.04776858538389206, -0.06170818582177162, -0.00561891682446003, -0.004554451908916235, 0.015333268791437149, 0.007577758748084307, -0.05042300745844841, -0.0847509428858757, 0.08090105652809143, -0.013421581126749516, 0.06611697375774384, 0.06300456076860428, -0.005324768368154764, 0.005229654256254435, -0.08634745329618454, -0.013242645189166069, -0.04599800333380699, 0.05403725430369377, -0.012949350290000439, 0.014337356202304363, 0.0041498830541968346, 0.0606609471142292, -0.08193773031234741, 0.017858099192380905, -0.051886193454265594, 0.07952187955379486, -0.061364080756902695, -0.025713196024298668, -0.016789881512522697, 0.014740819111466408, -0.03425605222582817, -0.04095042124390602, 0.012873866595327854, -0.05306076630949974, -0.0869879350066185, -0.07677863538265228, 0.03756987303495407, 0.061609119176864624, -0.02917551063001156, 0.005450559314340353, -0.038036786019802094, 0.0020801906939595938, 0.07193658500909805, 0.023902229964733124, 0.009960716590285301, 0.07076369225978851, -0.02590472251176834, -0.03491010516881943, -0.0589798279106617, 0.015065532177686691, -0.020799733698368073, 0.03754435479640961, -0.036185018718242645, -0.08674032241106033, 0.059365298599004745, -0.055955853313207626, 0.042882196605205536, 0.061578169465065, -0.011756636202335358, -0.0714801624417305, -0.009801606647670269, 0.033124811947345734, -0.020871711894869804, 0.04144389182329178, -0.030951611697673798, -0.06748555600643158, -0.06728744506835938, 0.08162327855825424, -0.05497705563902855, 0.014006576500833035, 0.0710931047797203, -0.07010272890329361, -0.05400462448596954, 0.036335539072752, -0.018331730738282204, 0.021124983206391335, -0.006624502129852772, 0.006909617222845554, -0.029713910073041916, 0.030219025909900665, -0.021786492317914963, 0.044124945998191833, 0.07182326912879944, -0.08931401371955872, 0.052432332187891006, -0.04316047206521034, -0.027164814993739128, -0.08462200313806534, 0.05862486734986305, -0.09607399255037308, -0.04770669341087341, -0.07016216218471527, -0.03419072926044464, -0.0399010144174099, 0.05891389027237892, -0.09733200818300247, -0.09109311550855637, -0.040733423084020615, -0.0328851118683815, -0.03739326447248459, -0.0736699178814888, 0.015822624787688255, 0.051613420248031616, -0.05014198645949364, -0.008929532021284103, -0.03629959374666214, 0.030214831233024597, 0.043027035892009735, -0.014396963641047478, 0.019316675141453743, -0.09035813063383102, -0.04738203063607216, 0.002878806786611676, 0.017901722341775894, 0.05545995756983757, 0.031995367258787155, -0.0026923080440610647, -0.04438658431172371, -0.004751679487526417, -0.0383884496986866, -0.10785150527954102, 0.047853272408246994, 0.072202667593956, -0.018385447561740875, 0.01677887514233589, 0.09059357643127441, 0.06448099762201309, -0.014059184119105339, -0.038100466132164, -0.08392184972763062, -0.008751394227147102, 0.04135891795158386, -0.0859140008687973, -0.0410166010260582, -0.027659937739372253, -0.09737387299537659, -0.002789329970255494, -0.04254794865846634, 0.05553323030471802, 0.04128636047244072, 0.09966170787811279, -0.05091935396194458, 0.010626510716974735, -0.08374130725860596, -0.044187191873788834, 0.08584338426589966, -0.08786100894212723, -0.025870341807603836, -0.008467769250273705, -0.10966670513153076, -0.025364473462104797, 0.01265784166753292, -0.06603774428367615, -0.06660888344049454, -0.039248447865247726, 0.01645573228597641, -0.06403234601020813, -0.04244387149810791, -0.09896223247051239, 0.03543432801961899, -0.07549933344125748, -0.04524535685777664, -0.012780103832483292, 0.08686719089746475, -0.09695478528738022, -0.05514153093099594, -0.10549641400575638, -0.037602368742227554, 0.008681355975568295, -0.05963645502924919, 0.06279804557561874, 0.0019761889707297087, -0.054301872849464417, 0.04549134522676468, -0.051510460674762726, -0.07655884325504303, -0.015828125178813934, 0.0810651034116745, -0.051550958305597305, 0.032521799206733704, 0.00849142950028181, -0.02425472065806389, -0.02297363057732582, -0.05568091198801994, -0.053879573941230774, -0.09524007141590118, -0.06937222182750702, 0.015149341896176338, -0.03376835584640503, 0.03471432253718376, -0.036928940564394, -0.014073814265429974, 0.09267956018447876, -0.07803911715745926, -0.023682434111833572, 0.04928576573729515, 0.025444289669394493, -0.04853466898202896, 0.09843486547470093, -0.035293880850076675, -0.047870684415102005, -0.05002019554376602, 0.04799124598503113, -0.03820907324552536, -0.09665101021528244, 0.08917290717363358, -0.09618785977363586, -0.029972581192851067, -0.04010751470923424, 0.002547066891565919, 0.059533048421144485, -0.08433091640472412, -0.0010157458018511534, -0.020750505849719048, 0.07296691834926605, -0.05277441814541817, -0.08609379827976227, -0.0392809621989727, 0.08237660676240921, 0.0892297774553299, -0.005066285841166973, -0.07975689321756363, -0.07557950168848038, 0.02116100862622261, -0.0725996270775795, -0.00025916186859831214, -0.03102724626660347, 0.07392849028110504, -0.04821937903761864, -0.07558118551969528, -0.036580171436071396, 0.07749096304178238, 0.08462094515562057, 0.02925567887723446, -0.07647299766540527, 0.0703175961971283, -0.03648677468299866, 0.04059836268424988, -0.007339539006352425, -0.02093571424484253, -0.004172101151198149, -0.023792292922735214, -0.01662622205913067, -0.04944692552089691, 0.030746402218937874, 0.02638174779713154, 0.05609995871782303, 0.005904531572014093, 0.002223384566605091, -0.06101734936237335, 0.03253956511616707, 0.07834197580814362, -0.0713072270154953, 0.028952540829777718, -0.05908162146806717, 0.02807803452014923, -0.0024289803113788366, 0.011181681416928768, 0.07945262640714645, 0.09442205727100372, -0.0014160918071866035, -0.06725968420505524, -0.026595208793878555, 0.09796912223100662, -0.04972905293107033, 0.04427328705787659, -0.017607668414711952, 0.08311575651168823, -0.02640758641064167, 0.03435554727911949, 0.08258579671382904, -0.02345932647585869, -0.04506363719701767, 0.0363214910030365, 0.07439183443784714, -0.1066320464015007, 0.0656951293349266, 0.012457970529794693, -0.07469406723976135, -0.04500522464513779, 4.391823313198984e-05, -0.07843086868524551, 0.016669461503624916, -0.08211289346218109, 0.030500616878271103, -0.049261219799518585, -0.04528694599866867, -0.010193987749516964, 0.07413000613451004, -0.07956340163946152, 0.07302706688642502, -0.016010383144021034, 0.012762441299855709, -0.024516429752111435, -0.004796865861862898, 0.0529191680252552, 0.08326984196901321, 0.09959804266691208, -0.03826408460736275, -0.05335622280836105, -0.043678127229213715, 0.0649695098400116, -0.06745974719524384, 0.004135537426918745, 0.0279010608792305, 0.11734534800052643, -0.016060402616858482, 0.05181489139795303, 0.011143079027533531, -0.001773185096681118, 0.03326079249382019, -0.0688009187579155, -0.008545361459255219, 0.03576153144240379, 0.07139333337545395, -0.07586345076560974, -0.01479319017380476, 0.043741777539253235, 0.08892330527305603, 0.046519793570041656, 0.05822863057255745, -0.0028727904427796602, -0.012198470532894135, -0.044280603528022766, -0.010157296434044838, 0.09844975918531418, -0.04848161339759827, 0.06643413007259369, -0.02379993349313736, 0.03209254890680313, -0.061480797827243805, -0.07270938903093338, 0.07413569837808609, 0.012322557158768177, 0.0875471979379654, -0.052733637392520905, -0.05385933816432953, 0.0050697629339993, -0.10534579306840897, 0.03651249408721924, 0.05779287591576576, 0.0869758203625679, -0.030907073989510536, -0.044771693646907806, 0.05700245872139931, 0.05250052362680435, 0.059270426630973816, 0.01184757798910141, -0.024241110309958458, 0.032035406678915024, -0.04374033957719803, -0.07887628674507141, -0.01960415579378605, 0.04239129275083542, 0.015540431253612041, 0.015518121421337128, -0.043051544576883316, 0.025998707860708237, -0.07537832856178284, 0.02953943982720375, -0.07367808371782303, 0.05556200072169304, 0.0767301544547081, -0.02148674987256527, -0.03189576789736748, -0.02392972819507122, -0.07645213603973389, 0.03634677454829216, 0.01842682808637619, 0.02259240113198757, 0.1054837629199028, 0.0586431510746479, 0.014099233783781528, -0.08332731574773788, 0.010158581659197807, -0.0890432819724083, 0.010490482673048973, 0.07338797301054001, 0.02528858371078968, -0.06225298345088959, 0.04924764856696129, 0.06919344514608383, -0.014047607779502869, -0.03306209295988083, 0.06833707541227341, -0.027935171499848366, 0.037952542304992676, 0.07990244776010513, -0.009198150597512722, 0.04259147867560387, 0.02302003651857376, 0.04018578678369522, 0.060102224349975586, 0.02822098135948181, -0.004809882026165724, 0.015014723874628544, -0.036832187324762344, 0.05478416010737419, 0.039269980043172836, 0.059128984808921814, 0.018154265359044075, 0.08213739842176437, -0.052564121782779694, -0.050444331020116806, -0.0825781598687172, -0.10185790061950684, -0.027599724009633064, 0.08057921379804611, 0.020615411922335625, -0.10229118168354034, -0.09000709652900696, 0.05446462333202362, -0.06787238270044327, -0.040222566574811935, 0.04537733271718025, 0.10211223363876343, 0.026455964893102646, 0.09069854766130447, 0.00929321814328432, -0.04456658288836479, -0.058769673109054565, 0.032751794904470444, 0.06300447881221771, 0.02477867901325226, -0.07515062391757965, 0.05300799012184143, -0.04850463196635246, -0.027390673756599426, 0.052792150527238846, -0.06890467554330826, 0.03541020676493645, 0.023884091526269913, 0.03832211717963219, -0.049502234905958176, 0.027604106813669205, -0.020276201888918877, 0.07300912588834763, 0.05417428910732269, -0.04062548279762268, 0.061076026409864426, 0.014843285083770752, 0.026290694251656532, -0.0945487916469574, -0.03463474288582802, -0.06576338410377502, 0.03289211913943291, 0.056250765919685364, -0.015367736108601093, -0.01999165304005146, 0.017449727281928062, 0.06192132458090782, -0.022511985152959824, -0.043931085616350174, -0.012662246823310852, 0.07710325717926025, -0.08928517997264862, -0.04292500764131546, -0.08503974229097366, -0.041672755032777786, -0.0013508028350770473, -0.03345043584704399, 0.017623594030737877, -0.018839402124285698, 0.018518539145588875, 0.07369211316108704, -0.1166696771979332, -0.03316324204206467, -0.06603240221738815, 0.03323125094175339, 0.05699329078197479, -0.05664468929171562, 0.025532487779855728, 0.046552494168281555, 0.06645403057336807, 0.04987267777323723, -0.09585165977478027, -0.06235399469733238, 0.050353214144706726, -0.002110977889969945, -0.08079736679792404, -0.018109364435076714, -0.018083106726408005, 0.08480484038591385, -0.05987820401787758, 0.08606410771608353, -0.0020973829086869955, 0.0537150576710701, 0.05148513615131378, -0.05993293225765228, -0.05664205178618431, 0.0486145056784153, -0.02874895930290222, -0.04739353805780411, 0.06508009135723114, 0.0823456421494484, -0.03709673136472702, -0.035309914499521255, 0.06104998290538788, 0.0792827159166336, 0.06207134202122688, -0.059268414974212646, 0.05261531472206116, 0.015141811221837997, -0.03541531041264534, 0.045157015323638916, -0.05972166731953621, -0.057671695947647095, 0.023265322670340538, -0.08045820146799088, -0.02999795228242874, -0.042718466371297836, -0.010005105286836624, 0.05012378841638565, 0.08115077763795853, 0.008068940602242947, 0.0047022495418787, -0.06312904506921768, 0.011425883509218693, 0.10439853370189667, -0.06075634807348251, -0.05962466076016426, -0.06209610775113106, -0.05952812731266022, 0.046759266406297684, -0.08474184572696686, 0.02826845273375511, 0.022739363834261894, 0.06890206038951874, -0.050364233553409576, -0.03602723777294159, 0.013457613997161388, 0.02267466112971306, 0.01365647278726101, 0.032384518533945084, -0.0717737004160881, 0.06673773378133774, -0.11104544252157211, 0.0008328748517669737, -0.03567107021808624, 0.07746820896863937, 0.060754869133234024, -0.0720585361123085, -0.04407177120447159, -0.0358029380440712, -0.05748053640127182, -0.019455473870038986, -0.06631652265787125, -0.025079645216464996, -0.03505243733525276, -0.04323295131325722, -0.0633445605635643, 0.010667302645742893, 0.046694427728652954, 0.054074399173259735, 0.08113857358694077, 0.016445796936750412, 0.07394678890705109, 0.07877980917692184, 0.06345178931951523, -0.03507724031805992, -0.03876999765634537, 0.009034077636897564, 0.0223524272441864, 0.04622874781489372, -0.04131542891263962, 0.00496186688542366, 0.0835924968123436, -0.055521465837955475, 0.05938228964805603, -0.005881043151021004, 0.10134189575910568, 0.04647981375455856, -0.04525899514555931, -0.020957373082637787, 0.05270064249634743, 0.017832061275839806, -0.0799257904291153, -0.06248526647686958, 0.04390689730644226, 0.046350229531526566, 0.03457950800657272, -0.04837309569120407, 0.05607903376221657, -0.016181234270334244, 0.01877451315522194, -0.07945264875888824, -0.062118858098983765, -0.03990061208605766, 0.031659163534641266, 0.03859732300043106, -0.020859897136688232, -0.01782519929111004, 0.03480064496397972, 0.046675778925418854, -0.0779549777507782, 0.020577188581228256, 0.01955314166843891, -0.024402474984526634, -0.004069661721587181, -0.05670098960399628, -0.031175309792160988, -0.07389308512210846, -0.06961456686258316, 0.048396866768598557, 0.06174980476498604, -0.030974116176366806, -0.030609313398599625, 0.010152881033718586, 0.01044677197933197, -0.00016134887118823826, 0.09386969357728958, 0.051687028259038925, 0.0036990433000028133, -0.07407518476247787, -0.04260410740971565, 0.02024405263364315, 0.016526278108358383, -0.047326091676950455, 0.029086068272590637, 0.04202105477452278, -0.04626144841313362, 0.07472530007362366, -0.0026889049913734198, 0.040948957204818726, 0.05709630995988846, -0.031876660883426666, 0.042908743023872375, 0.09161543101072311, -0.09333748370409012, -0.02504867874085903, -0.06703329086303711, -0.02411949262022972, -0.09050469845533371, -0.03282158076763153, -0.08687175065279007, 0.0015814956277608871, 0.03659223020076752, 0.04118524491786957, -0.08753403276205063, 0.0316493958234787, -0.03975839540362358, -0.07091838121414185, 0.02889207936823368, -0.024153416976332664, -0.05576656758785248, -0.054576192051172256, -0.07307004183530807, 0.045811790972948074, -0.009497420862317085, 0.08088545501232147, 0.036665428429841995, 0.046318139880895615, 0.007454754784703255, 0.041497211903333664, -0.013704926706850529, -0.08414488285779953, 0.10155924409627914, 0.007760855369269848, -0.07036340981721878, 0.10587873309850693, -0.06901773065328598, -0.05108391493558884, 0.024909429252147675, -0.06657582521438599, 0.09512576460838318, 0.035727228969335556, 0.08265922963619232, -0.0869680866599083, -0.06396165490150452, -0.0003579907934181392, -0.03609725460410118, -0.027480660006403923, -0.04927784577012062, 0.09444314986467361, -0.04602089151740074, -0.09410153329372406, 0.04386885464191437, -0.06134229525923729, 0.028103511780500412, 0.04226147383451462, -0.09012463688850403, 0.004925456829369068, -0.00917050987482071, -0.0907265767455101, -0.05158660560846329, -0.02773822471499443, 0.08873138576745987, 0.050147149711847305, 0.1084209457039833, -0.036240674555301666, -0.02784176543354988, 0.009048040956258774, 0.058401454240083694, -0.04827171191573143, -0.04159806668758392, 0.023238033056259155, 0.04145774990320206, 0.0748884528875351, -0.05190642178058624, -0.07099318504333496, -0.07968275994062424, -0.04655376449227333, -0.06455296277999878, 0.0793793573975563, 0.0072391461580991745, 0.05668613314628601, 0.0455809012055397, 0.005151534453034401, -0.03011520579457283, -0.04537447541952133, -0.07656030356884003, -0.007816520519554615, 0.054212722927331924, 0.02498665079474449, 0.08143414556980133, -0.024155268445611, 0.05429792404174805, -0.009628618136048317, 0.09629789739847183, 0.0204976424574852, 0.026108209043741226, 0.06094592437148094, -0.050177596509456635, -0.03451159968972206, -0.06369569152593613, -0.006475862115621567, 0.036366723477840424, 0.006447499617934227, -0.10776703804731369, 0.017112601548433304, 0.07509973645210266, 0.05529363825917244, -0.08047384768724442, -0.021641502156853676, 0.017086537554860115, 0.06793995946645737, 0.06231353431940079, -0.02591979317367077, 0.002765755169093609, 0.05167054012417793, -0.003810023656114936, -0.08393091708421707, 0.04143431410193443, -0.03936308994889259, -0.07223538309335709, 0.019627101719379425, 0.03112402744591236, 0.03318887948989868, 0.06048218533396721, -0.028415393084287643, -0.016286015510559082, -0.06512308120727539, 0.08020883798599243, 0.08316503465175629, 0.054930780082941055, -0.07208675891160965, 0.0997563824057579, -0.01228890847414732, 0.03220069780945778, -0.0511249415576458, 0.08797474950551987, -0.1136278510093689, -0.07818567752838135, -0.10879334062337875, 0.012542935088276863, 0.08092988282442093, -0.017256570979952812, 0.03523940220475197, 0.054666001349687576, 0.08973569422960281, -0.04977728798985481, -0.03121752291917801, 0.05764374881982803, 0.07703876495361328, 0.0046620662324130535, 0.09045993536710739, 0.0057583278976380825, -0.01868845708668232, 0.03533237800002098, 0.011223038658499718, 0.02287716418504715, 0.06912794709205627, 0.05506591871380806, 0.05873554199934006, 0.05710192397236824, 0.0905996784567833, 0.07223322242498398, -0.016070162877440453, -0.01980232074856758, -0.0012364047579467297, 0.016315655782818794, -0.09769608825445175, 0.04849163815379143, 0.09323254227638245, 0.03702544793486595, 0.060753293335437775, 0.031864941120147705, -0.017634766176342964, 0.0028383934404700994, 0.008657385595142841, 0.09054619073867798, -0.07442695647478104, 0.0032577477395534515, 0.0567588284611702, 0.049605656415224075, -0.031894925981760025, 0.08614660054445267, 0.051129311323165894, 0.07203812152147293, 0.007635687477886677, 0.02973731979727745, -0.07165732979774475, 0.05807232856750488, -0.020152278244495392, 0.03703155368566513, 0.056077439337968826, -0.00163573760073632, -0.06027866154909134, -0.04287942126393318, -0.0062476396560668945, -0.03689693287014961, -0.1033768430352211, -0.07721647620201111, 0.02627495862543583, -0.060731664299964905, -0.03909016028046608, 0.008258285000920296, 0.03380410373210907, -0.020790286362171173, 0.08031641691923141, -0.10845287144184113, 0.07264139503240585, -0.011596755124628544, 0.03107372485101223, -0.025069160386919975, -0.0016641666879877448, -0.06713157892227173], "bias": [0.04159926250576973, -0.07615373283624649, -0.03499794751405716, -0.08208749443292618, 0.064272440969944, -0.004196965601295233, -0.06101389229297638, -0.007019754033535719, 0.04352862015366554, -0.04570748284459114, -0.020064836367964745, -0.09915183484554291, 0.0038724280893802643, 0.05051498860120773, -0.07595261931419373, -0.13025681674480438, -0.026510171592235565, -0.005010861437767744, -0.03524402156472206, -0.02666034735739231, -0.0890885442495346, -0.022918161004781723, 0.10473934561014175, 0.07643477618694305, 0.018662991002202034, 0.0003740225511137396, 0.04908683896064758, -0.036048855632543564, -0.019411690533161163, -0.08433342725038528, 0.08962439000606537, -0.09516546875238419, -0.0187093336135149, 0.014167147688567638, -0.021029233932495117, -0.11801178008317947, -0.1008240282535553, -0.018780961632728577, -0.031631261110305786, -0.05689023807644844, -0.03687890246510506, -0.03709772229194641, -0.08579797297716141, 0.03432755172252655, -0.03593623265624046, -0.09431348741054535, 0.02767086587846279, 0.04603716358542442, 0.006687113083899021, -0.12601499259471893, -0.023779846727848053, 0.024006638675928116, 0.10681517422199249, 0.009379910305142403, -0.03803307190537453, -0.01552800927311182, 0.013893780298531055, 0.0948314368724823, 0.0638420358300209, 0.044168565422296524, -0.04819817095994949, 0.0031609039288014174, -0.016386372968554497, 0.008721922524273396, -0.009192418307065964, -0.008093059062957764, 0.027375800535082817, 0.03987522050738335, 0.009690746665000916, 0.026364514604210854, 0.04134674370288849, -0.04728444293141365, -0.048186007887125015, 0.07512319087982178, -0.05207853019237518, 0.05033465847373009, -0.024727459996938705, 0.06694123893976212, -0.0433010570704937, -0.006923038978129625, 0.07559549808502197, -0.11384151875972748, -0.07785766571760178, -0.010045510716736317, -0.0894165188074112, -0.05125247687101364, 0.07065979391336441, 0.02080332674086094, -0.08288589864969254, 0.08808084577322006, -0.05132828652858734, 0.07924766838550568, 0.009061277844011784, 0.07205653190612793, 0.05502616614103317, -0.053992860019207, -0.09779266268014908, 0.05801734700798988, -0.08172629028558731, -0.05633668601512909, -0.07141787558794022, 0.019290000200271606, -0.12142615765333176, -0.012600324116647243, -0.008350970223546028, -0.003572450950741768, 0.050869133323431015, -0.08878213912248611, -0.01980696991086006, 0.07061932981014252, -0.008691833354532719, 0.0033225121442228556, 0.045934539288282394, -0.04550779610872269, -0.022037962451577187, 0.01580812968313694, 0.0290926992893219, -0.06845815479755402, 0.041855257004499435, -0.04441601783037186, 0.0518249087035656, 0.03728242218494415, 0.03658360615372658, -0.16939564049243927, 0.05828474089503288, 0.029238905757665634, 0.06201522797346115, 0.04060087352991104]}, {"kind": "relu"}, {"kind": "affine", "scale": [2.1417369842529297, 2.2555184364318848, 1.9662960767745972, 2.4681577682495117, 2.494697332382202, 2.711836814880371, 2.7136952877044678, 1.8768330812454224, 2.0705718994140625, 1.897254467010498, 2.7847321033477783, 2.468501329421997, 2.197061777114868, 2.211078643798828, 2.7020087242126465, 2.659385919570923, 2.1792242527008057, 2.4206933975219727, 2.6998720169067383, 2.210190534591675, 3.01719331741333, 2.547332525253296, 2.4283294677734375, 1.8833624124526978, 2.5126442909240723, 2.226160764694214, 2.0513062477111816, 2.1810145378112793, 2.4078056812286377, 1.9505335092544556, 2.488987445831299, 2.544588327407837, 2.9595305919647217, 2.8788511753082275, 2.3166277408599854, 3.188615560531616, 2.675532817840576, 1.9554510116577148, 2.6743831634521484, 2.5301878452301025, 2.643120527267456, 2.633950710296631, 2.7832839488983154, 2.441622018814087, 2.237168312072754, 1.7183454036712646, 1.7945340871810913, 2.2581000328063965, 1.9527822732925415, 2.930575370788574, 2.6853444576263428, 2.3882029056549072, 2.280848979949951, 2.4587090015411377, 2.319931745529175, 1.9944947957992554, 2.372290849685669, 2.2113165855407715, 1.970880150794983, 1.8951208591461182, 2.6921474933624268, 2.1764471530914307, 1.589803695678711, 1.881813406944275, 2.3623108863830566, 2.6089539527893066, 2.023545742034912, 2.0954277515411377, 2.1016077995300293, 2.7899458408355713, 2.311527729034424, 2.3876256942749023, 2.421757221221924, 2.378147602081299, 2.8988873958587646, 2.098398447036743, 2.2976324558258057, 1.9824628829956055, 1.9158064126968384, 2.621250867843628, 2.1900153160095215, 3.0936689376831055, 2.299819231033325, 2.016977548599243, 2.1522789001464844, 2.304154872894287, 2.6265978813171387, 2.0458219051361084, 2.391505241394043, 2.615597724914551, 1.971319317817688, 1.933479905128479, 2.0127811431884766, 2.235707998275757, 2.6172173023223877, 2.407900094985962, 3.22599458694458, 2.1070704460144043, 2.706852674484253, 2.6986677646636963, 2.4680845737457275, 2.2303552627563477, 2.832267999649048, 2.7477128505706787, 2.3663430213928223, 2.1083264350891113, 1.7015783786773682, 1.8606765270233154, 2.5383474826812744, 2.287503480911255, 2.427999496459961, 2.411720037460327, 2.355644702911377, 2.5468201637268066, 2.3941845893859863, 2.293837308883667, 1.9530428647994995, 2.3257100582122803, 2.478320360183716, 2.7429521083831787, 2.7287802696228027, 2.581552505493164, 2.5320205688476562, 3.450134515762329, 2.155653715133667, 2.31396746635437, 2.0637452602386475, 2.0461513996124268], "bias": [-0.6681289672851562, -0.5996702909469604, -0.6112561225891113, -0.5813490748405457, -0.837609589099884, -0.6828917860984802, -0.7161933183670044, -0.5949373245239258, -0.6341429948806763, -0.5947930812835693, -0.748478889465332, -0.5351269245147705, -0.6690796613693237, -0.7563871145248413, -0.6776052713394165, -0.5729687213897705, -0.6248465776443481, -0.7496203184127808, -0.7790960073471069, -0.6829865574836731, -0.7265080213546753, -0.7078098058700562, -0.8177064657211304, -0.6487263441085815, -0.6900030374526978, -0.6168724298477173, -0.6275293231010437, -0.5739729404449463, -0.6562868356704712, -0.5732683539390564, -0.7964076399803162, -0.565880298614502, -0.7538550496101379, -0.7098776698112488, -0.6400022506713867, -0.6203922033309937, -0.6010755300521851, -0.5875330567359924, -0.6226927042007446, -0.684378445148468, -0.7814904451370239, -0.6620230078697205, -0.6805528402328491, -0.6817560195922852, -0.5691108703613281, -0.563885509967804, -0.6045318245887756, -0.6771635413169861, -0.692303478717804, -0.5877575874328613, -0.7060989141464233, -0.7166895866394043, -0.8587713241577148, -0.6824492812156677, -0.6809839010238647, -0.6758059859275818, -0.7001523971557617, -0.7941382527351379, -0.6165275573730469, -0.6595649123191833, -0.7010179162025452, -0.6325765252113342, -0.5301017761230469, -0.5540369749069214, -0.6648423075675964, -0.7404164671897888, -0.7124212980270386, -0.6482963562011719, -0.6531479954719543, -0.7596098184585571, -0.6007802486419678, -0.5896417498588562, -0.6136000752449036, -0.8768556714057922, -0.67901211977005, -0.6097642779350281, -0.6062843203544617, -0.7968437671661377, -0.5576589107513428, -0.6363639831542969, -0.8419266939163208, -0.5569130778312683, -0.5765963792800903, -0.6273539066314697, -0.574666440486908, -0.6078649759292603, -0.8174697160720825, -0.6887447237968445, -0.5639398097991943, -0.8093569278717041, -0.5536355376243591, -0.6933518052101135, -0.628250002861023, -0.745573103427887, -0.7851549386978149, -0.527691662311554, -0.6146597266197205, -0.6996747255325317, -0.6724056005477905, -0.5645427107810974, -0.5876831412315369, -0.5880085229873657, -0.5987040996551514, -0.7554624080657959, -0.6068323850631714, -0.6309123039245605, -0.6189883351325989, -0.5461612343788147, -0.5930984616279602, -0.6988772749900818, -0.6241075992584229, -0.7911058664321899, -0.6705461144447327, -0.7439833283424377, -0.6139439344406128, -0.6222403049468994, -0.6377288103103638, -0.6120184659957886, -0.7601687908172607, -0.7724931240081787, -0.7345701456069946, -0.7706950902938843, -0.6717736721038818, -0.6455988883972168, -0.6796804070472717, -0.6792722940444946, -0.6327857971191406, -0.6532109975814819]}, {"kind": "linear", "input": 128, "output": 1, "weight": [0.07349225133657455, -0.06246498227119446, 0.10960866510868073, -0.03602907434105873, -0.040242750197649, -0.027397045865654945, -0.08285661041736603, 0.11285504698753357, -0.10505963116884232, 0.0706976130604744, -0.07288488000631332, 0.06436651945114136, 0.054686982184648514, -0.032937150448560715, -0.07805970311164856, 0.06161196902394295, 0.07912612706422806, -0.10586274415254593, -0.04710279032588005, -0.06399901211261749, -0.038740821182727814, -0.0614183209836483, -0.06988861411809921, 0.05086842551827431, 0.05384014919400215, 0.055464692413806915, 0.10018470883369446, 0.07810252904891968, -0.05869520828127861, 0.10677661001682281, -0.08140218257904053, 0.0763440728187561, -0.08305156975984573, -0.059307727962732315, -0.09555529057979584, -0.029263563454151154, -0.058036819100379944, 0.10673778504133224, -0.006905953865498304, -0.03562178462743759, -0.0926010012626648, -0.009289069101214409, -0.07586097717285156, 0.0006788703030906618, -0.04508514329791069, 0.10435748100280762, 0.09217650443315506, -0.07648593932390213, 0.07357316464185715, -0.05067984387278557, -0.019429704174399376, -0.11186337471008301, -0.07074061036109924, -0.06258068978786469, -0.047987088561058044, -0.07874494045972824, -0.01895153895020485, -0.05161868408322334, -0.06333161145448685, 0.10781240463256836, -0.06952999532222748, -0.0743996649980545, -0.04779207333922386, 0.06767125427722931, 0.03655605390667915, -0.020394299179315567, -0.07659516483545303, 0.027813946828246117, 0.11681103706359863, -0.07636238634586334, -0.018355833366513252, -0.05778154358267784, 0.008829028345644474, -0.04066396877169609, -0.05110180005431175, 0.07401779294013977, 0.0059036011807620525, -0.08835731446743011, 0.10523774474859238, -0.02698170207440853, -0.03981377184391022, 0.05393853411078453, 0.0650707259774208, 0.05790090933442116, 0.0733189806342125, 0.036329180002212524, -0.05854487791657448, 0.10252571105957031, 0.0953790619969368, -0.031215455383062363, 0.10541712492704391, 0.03698281571269035, 0.07633999735116959, -0.043858565390110016, -0.08102451264858246, 0.0454966239631176, -0.048610869795084, 0.0674540251493454, -0.09737233072519302, -0.040784746408462524, -0.10427756607532501, -0.05199979618191719, -0.04240282624959946, -0.08787176758050919, 0.08168616890907288, 0.06228993460536003, 0.0780576840043068, 0.0490192174911499, -0.04878677427768707, -0.10128617286682129, 0.019614018499851227, -0.050627849996089935, 0.057663824409246445, -0.08565255999565125, 0.07028879225254059, -0.02204267866909504, 0.06786075979471207, 0.04907417669892311, -0.07230158150196075, -0.04421527311205864, -0.0021649335976690054, -0.05063724145293236, 0.04001600667834282, -0.04097739979624748, 0.11144301295280457, 0.04410025104880333, 0.06443550437688828, 0.06826489418745041], "bias": [0.03734079375863075]}]} diff --git a/sound2view.toml b/sound2view.toml index dc59b6f..2aeb97a 100644 --- a/sound2view.toml +++ b/sound2view.toml @@ -42,10 +42,20 @@ f_hi_hz = 4100.0 grid_width = 64 # Focus distance in meters grid_z = 2.0 +# The scan grid reaches this many degrees beyond the field of view on every side (at most 55 +# degrees off axis); only the visible part is shown. A source outside the image gets its own cell +# there instead of lighting up the image border, and CLEAN-SC takes it out of the visible part. +# Map time grows with the cells (about 3x at 15); 0 turns it off. +grid_margin_deg = 15.0 +# Leave impulsive sounds (snapping, clapping, a hammer) out of the map entirely (toggle in the +# settings menu). Off, they are shown like any other source but can never be speech: a hand +# snapping next to a talker is inside the beam toward the talker, both beams carry the same audio, +# and only a second map without the snaps tells them apart +skip_transients = false [beamformer] # "das", "functional" or "clean_sc" (selectable in the GUI) -method = "functional" +method = "clean_sc" # Exponent of functional beamforming: larger is sharper, more sensitive to model errors nu = 20.0 # Functional: subtract the median eigenvalue (noise estimate) per bin (toggle in the GUI) @@ -62,17 +72,86 @@ clean_max_iter = 10 clean_stop_db = 20.0 # CLEAN-SC: width (standard deviation) of the Gaussian clean beam in grid cells; 0: single points clean_beam_cells = 1.5 +# CLEAN-SC: give every frequency bin with a source the same weight in the map (toggle in the +# settings menu). Otherwise the loudest bins decide, and those are the low ones, where the array +# cannot separate a source from its reflection off a wall or the ceiling +clean_equal_bins = true [display] # Overlay range in dB below the map maximum (adjustable in the GUI) db_range = 3.0 [peaks] -# Estimated source positions (blue dots): local maxima of the map (toggle in the GUI) +# Local maxima of each map, followed as tracks (see [tracking]) +# Show the tracks as dots (toggle "Tracks" in the GUI) enabled = true -# Only maxima at most this many dB below the map maximum -range_db = 6.0 +# Only maxima at most this many dB below the map maximum (a talker next to snapping is about +# 10 dB below it; whether a peak is a source at all is decided against the noise floor) +range_db = 12.0 # Minimum distance between two peaks in grid cells; the weaker one is dropped min_dist_cells = 4.0 # At most this many peaks, strongest first max_count = 5 +# A peak is a source only if this many dB more power arrives coherently from its direction than +# from any direction in silence (the noise floor, learned from the quietest map of the last +# minute: a pause between words is enough; a steady background source such as a fan is part of +# it). Slider next to "Tracks" in the GUI +min_above_floor_db = 8.0 +# Peaks closer than this to the border of the whole scan grid, in grid cells, are dropped (0: +# none). Set 1 if pipeline.grid_margin_deg is 0: then the grid ends at the image border, where +# sources outside the image have their maximum +edge_margin_cells = 0.0 +# Extra dB above the floor a source needs at the left and right image border (0 in the center, +# growing with the squared distance); central peaks are also preferred for max_count +center_weight_db = 2.0 +# Coherent share of the sound above 2 kHz a source needs, in dB (point sources and snaps about 0, +# diffuse noise far below; "Coherence" in the GUI) +min_coherence_db = -8.0 + +[tracking] +# A peak continues a track if it is at most this far away, in grid cells (noise maxima jump +# farther and keep starting new tracks) +max_dist_cells = 4.0 +# A track is shown (and recorded in the CLI modes) after a matching peak in min_hits of the last +# confirm_window maps (not necessarily in a row) +min_hits = 3 +confirm_window = 5 +# Maps in a row without a matching peak until a track is dropped +max_missed = 5 +# Weight of the new peak in the smoothed position, in (0, 1]; 1 follows the peak exactly (less +# puts the dot between two spots when the map's maximum jumps, e.g. to a reflection) +smoothing = 1.0 + +[speech] +# Classify the audio of each track (voice-activity-detection model); speech tracks are green +enabled = true +# Model exported by voice-activity-detection, relative to the working directory +model = "model/speech_detector.json" +# Smoothed probability at which a track becomes speech (adjustable in the GUI) +on_threshold = 0.95 +# Speech is held this long after the last frame at on_threshold or above, in ms (pauses between +# words stay speech; other sounds after the speech do not keep it) +hold_ms = 500.0 +# Time constant of the probability smoothing, in ms (0: no smoothing) +smoothing_ms = 50.0 +# The smoothed probability must stay at on_threshold this long before a track turns green, in ms +# (snapping and clapping are shorter than a syllable) +min_onset_ms = 80.0 +# A speech track whose audio correlates at least this much (within 20 ms) with a speech track that +# has echo_margin_db more speech energy repeats its speech (reflection, ghost, leakage into another +# sound such as snapping) and stays blue ("echo of #N"); 1 turns the check off +echo_correlation = 0.2 +echo_margin_db = 6.0 +# Speech ends at once when the track moves farther than this (grid cells) from where it was last +# clearly speech, so another source nearby that takes over the track does not inherit it +max_shift_cells = 3.0 +# Gain on the track audio before the classifier, in dB (the classifier expects microphone levels, +# speech around -25 dBFS; the UMA-16 gives about -60 dBFS) +gain_db = 36.0 +# Classifier frames quieter than this (after gain_db) count as non-speech, in dBFS +min_level_db = -55.0 +# A track is speech only while its share (dB) of the power received above 2 kHz that the other +# tracks do not explain is at least this: below 2 kHz every beam carries every talker, so a +# talker elsewhere (also outside the image) would turn other tracks green; a loud track next to +# the talker (snapping) does not hide the talker's share +min_source_db = -6.0