Elasticsearch 7.16.0 发布

Allocation

• Improve single-node watermark deprecation checks #80824 (issue: #80817)

Authentication

• Adding a deprecation info API check for SAML nameid_format setting #77276 (issues: #42404, #44090)

Cluster Coordination

• Adding a deprecation info check for cluster.join.timeout #76944 (issue: #42404)

Geo

• Adding deprecation checks for geo_shape parameters #76627 (issues: #42404, #70850)

ILM+SLM

• Add deprecation info and warnings for an empty _tier_preference #79305 (issue: #76147)
• Adding deprecation logging and deprecation info API for ILM freeze ation #77969 (issues: #42404, #70192)

Indices APIs

• Add frozen index deprecation check #78915 (issue: #70192)
• Deprecate ignore_throttled parameter #77479 (issue: #70192)
• Warn when searching a frozen index #78184 (issue: #70192)

Infra/Logging

• Change default deprecation logger level to CRITICAL #77030 (issue: #76754)
• Deprecation logs indexing is enabled by default #78991 (issues: #76292, #78850)
• Exposing the ability to log deprecated settings at non-critical level #79107 (issues: #77482, #78781)
• REST API to reset deprecation indexing cache #78392 (issue: #78134)
• Add category field to deprecation logs #77550 (issue: #67443)

Infra/Scripting

• Script: Deprecate script context cache #79508 (issue: #62899)

Java High Level REST Client

• Add deprecation headers to HLRC classes #79754

Machine Learning

• Deprecate estimated_heap_memory_usage_bytes and replace with model_size_bytes #80545

Monitoring

• Deprecate monitoring settings #79499
• Deprecate create_legacy_templates setting on HTTP exporters #78564
• Deprecate the monitoring ingest settings #77659 (issue: #77459)

Data streams

• API for adding and removing indices from a data stream #79279 (issue: #75852)

EQL

• Introduce repeatable queries #75082
• Refine repeatable queries #78895 (issue: #75082)

Machine Learning

• Text/Log categorization multi-bucket aggregation #71752

SQL

• Add ability to perform CCS through SQL querying #78903

Search

• Profile the fetch phase #77064 (issue: #75892)

Aggregations

• Adds support for the rate aggregation under a composite aggregation #76992 (issue: #76988)
• Interrupt aggregation reduce phase if the search task is cancelled #71714 (issue: #71021)
• Add deprecation check for adjacency matrix setting #77802 (issue: #46327)

Allocation

• Replace RoutingTable#shardsWithState(...) with RoutingNodes#unassigned(...) #78931 (issue: #77466)
• Reuse local node in async shard fetch responses #77991 (issue: #77266)

Audit

• Add cluster UUID and name to audit logs #79401 (issue: #72408)

Authentication

• Remove HTTPS check for API Keys & Service Accounts #76801

Authorization

• Add privileges to kibana_system for upgrading endpoint hidden indices #80140
• Allow fleet-server service account to set up Fleet #78192 (issue: #78078)
• Grant additional privileges for endpoint transform indices to kibana_system #79619
• Grant privileges required by package upgrade to kibana_system #78049 (issue: #77294)
• Optimize FLS/DLS setup in IndicePermission authz #77832
• Skip loading authorized indices if requests do not need them #78321
• Superuser fastpath for indexAccessControl #78498
• Update transform destination index privilege for kibana_system #79076
• Use fixed size memory allocation in IndicesPermission #77748
• Add proper permissions to fleet server for Endpoint response index #80231
• Migrate custom role providers to licensed feature #79127

CCR

• Add description to shard changes action request #80275 (issue: #79311)

Cluster Coordination

• Add cluster state serialization stats #78816
• Add timing stats to publication process #76771 (issue: #76625)
• Get hot threads on lagging nodes #78879
• Improve error message in 8.x to 7.x downgrade #78644 (issue: #78638)
• Improve logging in LeaderChecker #78883
• Mention “warn threshold” in master service slowlog #76815 (issue: #76625)
• Recycle pages used by outgoing publications #77407 (issue: #77317)
• Reuse previous indices lookup when possible #79004 (issues: #77888, #78980)
• Validate PING and STATE connections at join time #77741

Composite aggregations

• Support _first and _last ordering of missing values in composite aggregations #76740 (issues: #34550, #63523)

Data streams

• More accurate error message for data stream and alias #79027 (issues: #58327, #66163)

Distributed

• Add Fleet search API to wait on refreshes #73134 (issue: #71449)
• Add support for superseding in CancellableSingleObjectCache #80199

EQL

• Sequences will now support nano-timestamps #76953 (issue: #68812)

Engine

• Apply the reader wrapper on can_match source #78988
• Enable sort optimization in query Lucene changes #77907
• Honor max segment size when setting only_expunge_deletes on force merge #77478 (issues: #61764, #77270)

Geo

• Add support for metrics aggregations to mvt end point #78614 (issue: #77072)
• Include _index property for each hit in _mvt response #77995 (issue: #77205)
• Add track_total_hits support in mvt API #78074 (issue: #77222)

ILM+SLM

• Improve LifecycleExecutionState parsing #77855 (issue: #77466)
• Reduce the number of times that LifecycleExecutionState is parsed when running a policy #77863 (issue: #77466)
• Add built-in ILM policies for common user use cases #76791
• Allow for setting the total shards per node in the Allocate ILM action #76134

Indices APIs

• Fleet: Add action_response into .fleet-actions-results mapping #79584
• Store template’s mappings as bytes for disk serialization #78746
• Allow indices lookup to be built lazily #78745 (issue: #77466)

Infra/Core

• Add optional content checking to ResourceWatcher #79423
• Add other time accounting in HotThreads #79392
• Add upgrade preparation and completion callbacks to SystemIndexPlugin #78542
• Cache index.hidden setting #78612 (issue: #77974)
• Enable wait/blocked time accounting #77935 (issue: #72376)
• Handle empty /proc/self/cgroup file #78659 (issue: #77833)
• Implement framework for migrating system indices #78951
• Require System Index Descriptors to allow a specific suffix #78355
• Support mem type in nodes hot_threads API #72850 (issue: #70345)
• Use enum field for HotThreads report type #77462

Infra/Node Lifecycle

• Enable exit on out of memory error #71542

Infra/Scripting

• Add a direct sub classes data structure to the Painless lookup #76955
• Add ability to augment classes with fields from other classes in Painless #76628
• Add dynamic (duck) type resolution to Painless static types #78575
• Adds a lookup method to Painless for finding methods of all sub classes #77044
• UnsignedLong field type converter #77271
• Compile/cache eviction history metric placeholders #78257 (issue: #62899)

Infra/Settings

• Add show command to the keystore CLI #76693 (issue: #57261)
• Filtering setting deprecation info API messages based on a setting #78725

Ingest

• Add enrich node cache #76800 (issue: #48988)
• Add indices pipeline settings check when deleting a pipeline #77013
• Allow range types to be used for enrich matching #76110
• ECS support for Grok processor #76885 (issue: #66528)
• Improving cache lookup to reduce recomputing / searches #77259
• Make enrich policy execution cancelable #77188 (issue: #48988)
• Optimistic concurrency control for updating ingest pipelines #78551 (issue: #77031)
• Sync grok processor patterns with Logstash #76752
• Updating ingest pipeline without changes is no-op #78196 (issue: #77382)

Java High Level REST Client

• Add support for rest compatibility headers to the HLRC #78490 (issue: #77859)

License

• Add license family attribute to feature usage tracking #76622
• Add utility for tracking licensed persistent tasks #76672
• Separate feature usage tracking for FLS and DLS #79152
• Use a licensed feature per realm-type (+custom) #78810

Machine Learning

• Add new normalize_above parameter to p_value significant terms heuristic #78833
• Add new default char filter first_line_with_letters for machine learning categorization #77457
• Add new defer_definition_decompression parameter to put trained model API #77189 (issue: #77132)
• Enable ML on macOS on ARM #78203
• Track feature usage for data frame analytics, inference, and anomaly jobs #76789
• Speed up training of regression and classification models #2024
• Improve concurrency for training regression and classification models #2031
• Improve aspects of implementation of skip_model_update rule #2053
• Make sure instrumentation captures the best hyperparameters found for training classification and regression models https://github.com/elastic/ml-cpp/pull/2057{#2057}

Mapping

• Better error message for long keys in flattened fields #80433 (issue: #78248)
• Add time_series_metric parameter #76766 (issue: #74014)
• Add dimension mapping parameter #74450 (issue: #74014)

Monitoring

• Remove license check for monitoring data retention #79010

Packaging

• Switch to Ubuntu docker base image #80640
• Use Cloudflare’s zlib in Docker images #81245 (issue: #81208)
• Use almalinux as the Docker base image #80524 (issue: #76681)

Recovery

• Add support for peer recoveries using snapshots after primary failovers #77420 (issue: #73496)
• Respect generational files in recoveryDiff #77695 (issues: #55142, #55239)
• Limit concurrent snapshot file restores in recovery per node #79316 (issue: #79044)

Search

• Add _ignored and _routing metatada fields to fields API #78981 (issues: #75836, #78828)
• Add _index and _version metatada fields to fields API #79042 (issues: #75836, #78828)
• Add ability to retrieve _id via fields option #78828 (issue: #75836)
• Add node-level field capabilities API requests #79212 (issues: #74648, #77047, #78647)
• Add segment sorter for data streams #75195
• Add sort optimization with after from Lucene #64292
• Don’t always rewrite the Lucene query in search phases #79358
• Expand segment sorter for all timeseries indices #78639 (issue: #75195)
• Node level can match action #78765
• Search – return ignored field values from fields API #78697 (issue: #74121)
• Support request cache on frozen tier #77694 (issue: #75309)
• Use search_coordination threadpool for field capabilities API requests #79378 (issue: #79212)
• Create a sha-256 hash of the shard request cache key #74877 (issue: #74061)

Security

• Add extensionName() to security extension #79329
• Optimize StringMatcher for match-all patterns #77738
• CreateApiKey response now returns the base64-encoded credentials #77351 (issue: #50235)

Snapshot/Restore

• Add filtering by SLM policy to get snapshots API #77321
• Add sort by shard count and failed shard count to get snapshots API #77011
• Add descriptions to various tasks #76700
• Add maintenance service to clean up unused docs in snapshot blob cache #77686
• Add periodic maintenance task to clean up unused blob store cache docs #78438 (issue: #77686)
• Filter Unneeded SnapshotInfo Instances Early in TransportGetSnapshotsAction #78032 (issue: #74350)
• Implement exclude patterns for snapshot and repository names in get snapshots API #77308
• Implement sort by repository name in get snapshots API #77049
• Implement from_sort_value parameter in get snapshots API #77618

Stats

• Add cluster applier stats #77552
• Limit count of HTTP channels with tracked stats #77303
• Speedup computing cluster health #78969 (issue: #77466)
• Handle cgroups v2 in OsProbe #77128 (issues: #76812, #77126)

Transform

• Add _meta field to TransformConfig #79003 (issue: #77506)
• Add method to collect deprecations from a transform configuration #77565
• Add transform upgrade endpoint #77566
• Reduce indexes to query based on checkpoints #75839
• Implement the ability to preview the existing transform #76697 (issue: #76427)

Aggregations

• Add extra round trip to aggregation tests #79638 (issue: #73680)
• Fix rate aggregation with custom _doc_count #79346 (issue: #77734)
• Fix several potential circuit breaker leaks in aggregators #79676
• Scale doubles to floats when necessary to match the field #78344 (issue: #77033)
• Support for aggregation names with dots in first element path of a pipeline aggregation #77481

Allocation

• Make disk.threshold_enabled operator only #78822 (issue: #77846)

Authorization

• Improve permission granting for index referred by multiple names #78902
• Tighten API key behaviour with DLS and incompatible license #78378

CAT APIs

• Adjust _cat/templates to not request all metadata #78829

CCR

• Clear auto-follow errors on deleting pattern #80544 (issue: #77723)

CRUD

• Use query param instead of a system property for opting in for new cluster health response code #79351 (issues: #70849, #78940)

Cluster Coordination

• Avoid early release of local forking requests #77641 (issues: #77407, #77634)
• Check for global blocks after IndexNotFoundException in TransportMasterNodeAction #78128 (issue: #70572)
• Improve control of outgoing connection lifecycles #77295 (issue: #67873)
• Only remove active peer on connection failure #79557 (issues: #77295, #79550)
• Reduce merging in PersistedClusterStateService #79793 (issue: #77466)

CompositeAggs

• Fix composite aggregation tests failing after #76740 #77691 (issues: #76740, #77650)
• Revert 74559 (Avoid global ordinals in composite) #78846 (issues: #74559, #78836)

Data streams

• Add replicated field to get data stream API response #80988 (issue: #118899)
• Correct check for write index and increment generation on all data stream backing index operations #79916
• Fix data stream bug causing it to rollover to a non-existent Index #79759
• Fix IndexNotFoundException error when handling remove alias action #80312
• Fix data stream alias validation #81040 (issue: #80972)

Distributed

• Fix Fleet search API with no checkpoints #79400
• Modify Fleet search URLs to avoid URL collisions #79776

EQL

• Add optional fields and limit joining keys on non-null values only #79677

Geo

• Fix bug filtering collinear points on vertical lines #81155 (issues: #59501, #81076)
• Spherical mercator transformation should handle properly out of bounds latitudes #81145 (issue: #81128)
• Vector tiles: Add key bucket value to the aggregation layer #79634 (issue: #79585)

ILM+SLM

• Prevent duplicate ILM cluster state updates from being created #78390 (issues: #77466, #78246)
• Run ILM and SLM stopping cluster state updates at IMMEDIATE priority #80207 (issue: #80099)
• Validate that snapshot repository exists for ILM policies during GenerateSnapshotNameStep #77657 (issue: #72957)

Indices APIs

• Get templates APIs don’t support lists #78989 (issue: #78829)

Infra/Core

• Avoid spurious deprecation warnings when calling Deprecation Info API #78151 (issue: #78098)
• Fix overflow/underflow in CompositeBytesReference #78893
• Fix race condition in Feature Migration Status API #80572 (issue: #79680)
• Prevent stack overflow in rounding #80450
• Set LIBFFI_TMPDIR at startup #80651 (issues: #18272, #73309, #74545, #77014, #77053, #77285, #80617)
• Strip index blocks from settings for reindex targets #80887 (issue: #80654)
• Eschew leniency when parsing time zones #77267 (issues: #73955, #76415)
• Fix ingest timezone parsing #63876 (issue: #63458)

Infra/Logging

• Disable deprecation log indexing until templates are loaded #80406 (issue: #80265)

Infra/Settings

• Stricter UpdateSettingsRequest parsing on the REST layer #79228 (issue: #29268)
• Fix flood stage with system indices #80674

Ingest

• Addressing assertion failure, downgrading to exception – enrich #79717
• Fix executing missing enrich policy bug #80728

Java High Level REST Client

• Force typed keys in the HLRC get async search #78992 (issue: #77608)

Machine Learning

• Address potential ML feature reset permissions bug #79179
• Adjust ML memory tracker to reduce logging impact #78482
• Audit job open failures and stop any corresponding datafeeds #80665 (issues: #48934, #80621)
• Fix acceptable model snapshot versions in ML deprecation checker #81060 (issues: #79387, #81039, #119745)
• Fix autoscaling capacity consistency #81181
• Fix bug in inference stats persister for when feature reset is called #77213 (issue: #77182)
• Fix datafeed preview with remote indices #81099 (issue: #77109)
• Fix language identification bug when multi-languages are present #80675
• Fix model snapshot sorting when sorting by min_version #80596 (issue: #80561)
• ML legacy index templates that are no longer needed should be deleted #80874 (issue: #80876)
• Need to tolerate .ml-config being an alias #80025
• Parent datafeed actions to the datafeed’s persistent task #81143
• Wait for .ml-state-write alias to be readable #79731 (issue: #79636)
• Improve the estimates of hyperparameter importance in data frame analytics and avoid incorrectly stopping the hyperparameter search prematurely #2073
• Fix numerical instability in hyperparameter optimization for training regression and classification models #2078
• Fix numerical stability issues in time series modeling #[2083]

Mapping

• Disable request cache for non-deterministic runtime fields #75054
• Fix TextFieldMapper Retaining a Reference to its Builder #77251 (issue: #73845)

Packaging

• Ensure LIBFFI_TMPDIR is exported by init script #80794

Recovery

• Do not release snapshot file download permit during recovery retries #79409 (issue: #79316)

SQL

• Fix NULLS FIRST/LAST for aggregations #77750 (issue: #34550)
• Fix use of requestTimeout and pageTimeout query parameters #79360 (issue: #72151)
• Swap JDBC page.timeout and query.timeout properties in query requests #79491 (issue: #79430)

Search

• Minimize search source of shard level search requests #80634 (issue: #80187)
• Prevent NullPointerException in SourceConfirmedTextQuery #80472 (issue: #80419)
• _terms_enum API index_filter doesn’t work with _tier field on upgraded cluster #79553 (issue: #79200)
• Undeprecate the auto complete thread pool #80204
• Wildcard field regex query fix #78839 (issue: #78391)

Snapshot/Restore

• Fix Queued Snapshot Clone not Starting after Data Node Drops Out #77111 (issue: #77101)
• Fix Temporarily Leaking Shard Level Metadata Blobs in some Cases #76562
• Fix after restore Lucene.pruneUnreferencedFiles() conditional #81047 (issues: #68821, #75308)
• Improve handling of corrupt index.latest blob #77339
• Submit GCS delete batch requests incrementally #80540
• Fix repository-azure for empty settings on reload #79559

Transform

• Fix transform feature reset permissions bug #79178
• Respect timeout parameters in all APIs #79468 (issue: #79268)

Watcher

• Fix watcher check that determines when to serialize indices options #78070 (issue: #78035)
• Fix index action simulation when indexing several documents #76820 (issues: #66735, #74148)

Infra/Core

• Upgrade JNA to 5.10.0 #80617 (issue: #77014)

Watcher

• Update owasp-java-html-sanitizer dependency #80806

转自 https://www.elastic.co/guide/en/elasticsearch/reference/current/release-notes-7.16.0.html