This project is mirrored from https://github.com/cockroachdb/cockroach. Updated .
  1. 06 Jun, 2020 4 commits
    • craig[bot]'s avatar
      Merge #49802 · ef4eec09
      craig[bot] authored
      49802: opt: modify limit pushdown rule to support inner joins r=DrewKimball a=DrewKimball
      
      Previously, the optimizer could not push a limit into an InnerJoin.
      This patch replaces PushLimitIntoLeftJoin with two rules which
      perform the same function as well as handle the InnerJoin case.
      A limit can be pushed into a given side of an InnerJoin when rows
      from that side are guaranteed to be preserved by the join.
      
      Release note (sql change): improve performance for queries with a
      limit on a join that is guaranteed to preserve input rows.
      Co-authored-by: default avatarDrew Kimball <[email protected]>
      ef4eec09
    • craig[bot]'s avatar
      Merge #49788 · 1e836e6d
      craig[bot] authored
      49788: opt: add rules to eliminate a do-nothing join under a Project r=DrewKimball a=DrewKimball
      
      Previously, the optimizer couldn't eliminate a join in the input of a
      Project when the removal would have no effect on the output of the
      Project operator.
      
      This patch adds rules to replace a join with one of its input relations
      when the following conditions are met:
      
      1. The Project doesn't use any columns from the "discarded" input.
      2. The Join doesn't eliminate or duplicate rows from the "preserved" input.
      
      Fixes #49149
      
      Release note (sql change): The optimizer can now remove an unnecessary
      join from the input of a Project operator.
      Co-authored-by: default avatarDrew Kimball <[email protected]>
      1e836e6d
    • Drew Kimball's avatar
      opt: modify limit pushdown rule to support inner joins · ada13197
      Drew Kimball authored
      Previously, the optimizer could not push a limit into an InnerJoin.
      This patch replaces PushLimitIntoLeftJoin with two rules which
      perform the same function as well as handle the InnerJoin case.
      A limit can be pushed into a given side of an InnerJoin when rows
      from that side are guaranteed to be preserved by the join.
      
      Release note (sql change): improve performance for queries with a
      limit on a join that is guaranteed to preserve input rows.
      ada13197
    • Drew Kimball's avatar
      opt: add rules to eliminate a do-nothing join under a Project · 6bf047df
      Drew Kimball authored
      Previously, the optimizer couldn't eliminate a join in the input of a
      Project when the removal would have no effect on the output of the
      Project operator.
      
      This patch adds rules to replace a join with one of its input relations
      when the following conditions are met:
      
      1. The Project doesn't use any columns from the discarded input.
      2. The Join doesn't eliminate or duplicate rows from the preserved input.
      
      Fixes #49149
      
      Release note (sql change): The optimizer can now remove an unnecessary
      join from the input of a Project operator.
      6bf047df
  2. 05 Jun, 2020 23 commits
    • craig[bot]'s avatar
      Merge #49878 · dfb6e1be
      craig[bot] authored
      49878: roachtest: convert secondary-index-multi-version to version upgrade f… r=rohany a=rohany
      
      …ramework
      
      Fixes #49670.
      
      Release note: None
      Co-authored-by: default avatarRohan Yadav <[email protected]>
      dfb6e1be
    • craig[bot]'s avatar
      Merge #49851 · 77419519
      craig[bot] authored
      49851: rowcontainer: fix hash row container for some types r=yuzefovich a=yuzefovich
      
      The explanation is that `HashDiskRowContainer` is implemented using
      `DiskRowContainer` with the equality columns (i.e. the columns to hash)
      of the former being the ordering columns for the latter, and those
      ordering columns are used to compute the keys of the rows (in
      `encodeRow`) so that we could store the row in the sorted order. This
      way we store the build (right) side of the join, but for the probe
      (left) side we use `hashMemRowIterator` to compute the key of the
      probing row. The key computation methods must be the same in both
      places, otherwise, the results of the join can be incorrect. #45229
      broke this synchronization by changing the key computation method in
      `hashMemRowIterator.computeKey` to use `Fingerprint`. So we have to either
      use `Fingerprint` in `encodeRow` or use `Encode` in `computeKey`. The first
      choice doesn't seem to work because `Fingerprint` doesn't provide the
      ordering we need in `DiskRowContainer`, so we need to use the second approach.
      
      The ordering property is necessary because `DiskRowContainer` implements
      "hash row container" by sorting all rows on the ordering (i.e. hash) columns
      and using the ordering property to provide the "hashing" behavior (i.e. we
      would seek to the first row that has the same hash columns and then iterate
      from that row one row at a time forward until the hash columns remain the
      same).  If we don't have the ordering property, then the necessary invariant
      that all rows that hash to the same value are contiguous is not maintained.
      
      Release note: None
      Co-authored-by: default avatarYahor Yuzefovich <[email protected]>
      77419519
    • Rohan Yadav's avatar
      roachtest: convert secondary-index-multi-version to version upgrade framework · 28a136fd
      Rohan Yadav authored
      Fixes #49670.
      
      Release note: None
      28a136fd
    • craig[bot]'s avatar
      Merge #49761 · 432a1c53
      craig[bot] authored
      49761: colexec: add support for bit and some arithmetic binary operators r=yuzefovich a=yuzefovich
      
      **sem: unify division by zero check and fix it in a few places**
      
      Release note (bug fix): Previously, in some cases, CockroachDB didn't
      check whether the right argument of `Div` (`/`), `FloorDiv` (`//`),
      or `Mod` (`%`) operations was zero, so instead of correctly returning
      a "division by zero" error, we were returning `NaN`, and this is now
      fixed. Additionally, the error message of "modulus by zero" has been
      changed to "division by zero" to be inline with Postgres.
      
      **colexec: add support for bit and some arithmetic binary operators**
      
      This commit adds support for `Bitand`, `Bitor`, `Bitxor`, `FloorDiv`,
      and `Mod` binary operators for both native and datum-backed types.
      
      Release note (sql change): Vectorized execution engine now supports
      `Bitand` (`&`), `Bitor` (`|`), `Bitxor` (`^`), `FloorDiv` (`//`), and
      `Mod` (`%`) binary operators.
      Co-authored-by: default avatarYahor Yuzefovich <[email protected]>
      432a1c53
    • Yahor Yuzefovich's avatar
      colexec: add support for bit and some arithmetic binary operators · ca6c6649
      Yahor Yuzefovich authored
      This commit adds support for `Bitand`, `Bitor`, `Bitxor`, `FloorDiv`,
      and `Mod` binary operators for both native and datum-backed types.
      
      Release note (sql change): Vectorized execution engine now supports
      `Bitand` (`&`), `Bitor` (`|`), `Bitxor` (`^`), `FloorDiv` (`//`), and
      `Mod` (`%`) binary operators.
      ca6c6649
    • Yahor Yuzefovich's avatar
      sem: unify division by zero check and fix it in a few places · b973a823
      Yahor Yuzefovich authored
      Release note (bug fix): Previously, in some cases, CockroachDB didn't
      check whether the right argument of `Div` (`/`), `FloorDiv` (`//`),
      or `Mod` (`%`) operations was zero, so instead of correctly returning
      a "division by zero" error, we were returning `NaN`, and this is now
      fixed. Additionally, the error message of "modulus by zero" has been
      changed to "division by zero" to be inline with Postgres.
      b973a823
    • craig[bot]'s avatar
      Merge #49844 · 74e186e6
      craig[bot] authored
      49844: sql: tag skipped test with issue r=andreimatei a=andreimatei
      
      Release note: None
      Co-authored-by: default avatarAndrei Matei <[email protected]>
      74e186e6
    • craig[bot]'s avatar
      Merge #49849 · c9b9c01c
      craig[bot] authored
      49849: util/log: clean up and document the handling of stderr r=tbg a=knz
      
      Required by #48051
      
      Previous work in this area of the code introduced a confusion between
      two orthogonal concepts:
      
      - each logger might copy its log entries to the process' external
        stderr stream (e.g. the terminal during interactive use), as
        set by its "stderrThreshold" variable.
      
      - direct writes by Go code to process-wide file descriptor 2 (such
        as done by the Go runtime)  or `os.Stderr` (such as done
        by 3rd party packages when doing their own ad-hoc logging)
        can be redirected to a logger's output file.
      
      The confusion (mostly mine - @knz) was to mix the two kinds of
      "stderr" and mistakenly conflating "entries submitted to this logger
      via API calls" and "direct writes to fd 2 by other go code without
      looking at the logging API". These are actually completely separate
      and independent concepts/mechanisms.
      
      The code clarifies the situation as follows:
      
      - the process' external stderr stream is always available via
        `log.OrigStderr` and this is meant to never change throughout
        execution.
      
      - the external stderr stream is the sink for `Shout()` API calls
        and also the copy of log entries whose severity
        exceed the "stderrThreshold" variable.
      
      - separately, *at most one logger* may redirect internal writes to fd
        2 and os.Stderr to its log file. This is determined by
        its variable "noRedirectInternalStderrWrites" (previously
        named "noRedirectStderr").
      
      Beyond this, this commit fixes 3 bugs.
      
      1. the code was intending to both redirect the standard stderr file
         descriptor (fd 2 on unix, error handle on windows) and also
         `os.Stderr` separately, but failing to do so on unix build targets.
         It was done correctly for windows. This is now corrected so that
         `os.Stderr` gets assigned on all targets.
      
         (The separate assignment of `os.Stderr` is necessary because
         although Go intializes this to be equivalent to the standard
         file descriptor upon process startup, other Go code can
         assign `os.Stderr` after initialization.)
      
      2. upon encountering a write error to its output file, a logger
         would previously report that write error twice to the
         process' external stderr. This has been corrected
         so that the write error is only reported once.
      
      3. previously, upon a log.Fatal performed while configuration would not
         otherwise cause a copy the F entry go to the process external stderr,
         the code would override the configuration and force a copy of the
         entry to the external stderr stream.
      
         There is no good reason for this - either the user wants F entries
         on the external stderr and signal this desire via
         `--logtostderr=FATAL` or lower, or they don't want F entries there
         at all via `--logtostderr=NONE`. There is no reason for an
         override and the code should honor the configuration.
      
         (I introduced the bug together with the confusion mentioned
         at the beginning.)
      
      Release note: None
      Co-authored-by: default avatarRaphael 'kena' Poss <[email protected]>
      c9b9c01c
    • craig[bot]'s avatar
      Merge #49912 · d25c4638
      craig[bot] authored
      49912: kv,adminserver: properly remove the no-nodeid semantics r=tbg a=knz
      
      Fixes #49896
      
      The Decommission RPC doesn't need to accept an empty node ID list
      as of v20.2 since `cockroach quit --decommission` was removed.
      
      However, `TestDecommission` still used that case and started
      failing as a result. This patch fixes the test to not rely
      on the behavior.
      
      Additionally, this patch causes the RPC to return an error when no
      node ID is specified, instead of silently turning into a no-op.
      
      A discussion remains of whether the RPC should accept a way to specify
      the "local" node (maybe more explicitly than via an empty list of node
      IDs), like many of the other RPCs already do.
      
      This discussion came up in a separate issue which wants that
      behavior for the `node drain` command. I am expecting that `node
      decommission` will want that option too. However let's address that at
      that time.
      
      Release note: None
      Co-authored-by: default avatarRaphael 'kena' Poss <[email protected]>
      d25c4638
    • Raphael 'kena' Poss's avatar
      util/log: add an entry header incl timestamp to direct writes · ec7a7a25
      Raphael 'kena' Poss authored
      There's an uncommon code path that directly writes to either
      a log file or the process' external stderr:
      
      - to report I/O errors encountered during logging (e.g. errors
        writing to the output log file)
      - to copy the details of a panic object reported via `log.ReportPanic`
        to either external stderr or the log file, when the Go
        runtime would otherwise only report it to the "other side".
      
      In these cases, previously the data would be written as-is to the
      output sink, without a proper log entry header - timestamp, goroutine,
      file/line, etc.
      This made it hard to identify the precise moment when the
      output was produced and its precise origin.
      
      This patch enhances these cases by adding the missing log entry header.
      
      Release note: None
      ec7a7a25
    • Raphael 'kena' Poss's avatar
      util/log: clean up and document the handling of stderr · df3cee6c
      Raphael 'kena' Poss authored
      Previous work in this area of the code introduced a confusion between
      two orthogonal concepts:
      
      - each logger might copy its log entries to the process' external
        stderr stream (e.g. the terminal during interactive use), as
        set by its "stderrThreshold" variable.
      
      - direct writes by Go code to process-wide file descriptor 2 (such
        as done by the Go runtime)  or `os.Stderr` (such as done
        by 3rd party packages when doing their own ad-hoc logging)
        can be redirected to a logger's output file.
      
      The confusion (mostly mine - @knz) was to mix the two kinds of
      "stderr" and mistakenly conflating "entries submitted to this logger
      via API calls" and "direct writes to fd 2 by other go code without
      looking at the logging API". These are actually completely separate
      and independent concepts/mechanisms.
      
      The code clarifies the situation as follows:
      
      - the process' external stderr stream is always available via
        `log.OrigStderr` and this is meant to never change throughout
        execution.
      
      - the external stderr stream is the sink for `Shout()` API calls
        and also the copy of log entries whose severity
        exceed the "stderrThreshold" variable.
      
      - separately, *at most one logger* may redirect internal writes to fd
        2 and os.Stderr to its log file. This is determined by
        its variable "noRedirectInternalStderrWrites" (previously
        named "noRedirectStderr").
      
      Beyond this, this commit fixes 3 bugs.
      
      1. the code was intending to both redirect the standard stderr file
         descriptor (fd 2 on unix, error handle on windows) and also
         `os.Stderr` separately, but failing to do so on unix build targets.
         It was done correctly for windows. This is now corrected so that
         `os.Stderr` gets assigned on all targets.
      
         (The separate assignment of `os.Stderr` is necessary because
         although Go intializes this to be equivalent to the standard
         file descriptor upon process startup, other Go code can
         assign `os.Stderr` after initialization.)
      
      2. upon encountering a write error to its output file, a logger
         would previously report that write error twice to the
         process' external stderr. This has been corrected
         so that the write error is only reported once.
      
      3. previously, upon a log.Fatal performed while configuration would not
         otherwise cause a copy the F entry go to the process external stderr,
         the code would override the configuration and force a copy of the
         entry to the external stderr stream.
      
         There is no good reason for this - either the user wants F entries
         on the external stderr and signal this desire via
         `--logtostderr=FATAL` or lower, or they don't want F entries there
         at all via `--logtostderr=NONE`. There is no reason for an
         override and the code should honor the configuration.
      
         (I introduced the bug together with the confusion mentioned
         at the beginning.)
      
      Release note: None
      df3cee6c
    • Raphael 'kena' Poss's avatar
      kv,adminserver: properly remove the no-nodeid semantics · e0ad962a
      Raphael 'kena' Poss authored
      The Decommission RPC doesn't need to accept an empty node ID list
      as of v20.2 since `cockroach quit --decommission` was removed.
      
      However, `TestDecommission` still used that case and started
      failing as a result. This patch fixes the test to not rely
      on the behavior.
      
      Additionally, this patch causes the RPC to return an error when no
      node ID is specified, instead of silently turning into a no-op.
      
      A discussion remains of whether the RPC should accept a way to specify
      the "local" node (maybe more explicitly than via an empty list of node
      IDs), like many of the other RPCs already do.
      
      This discussion came up in a separate issues which wants that
      behavior for the `node drain` command. I am expecting that `node
      decommission` will want that option too. However let's address that at
      that time.
      
      Release note: None
      e0ad962a
    • craig[bot]'s avatar
      Merge #49905 · 326cb320
      craig[bot] authored
      49905: cliflags: fix the desc for the socket-dir flag r=tbg a=knz
      
      (found by @tbg)
      
      I had the description field mixed up when I previously
      deprecated `--socket` in favor of `--socket-dir`.
      
      Release note (bug fix): The description provided with `--help`
      on the CLI for `--socket-dir` and `--socket` has been fixed. They
      were invalid since v20.1.0.
      Co-authored-by: default avatarRaphael 'kena' Poss <[email protected]>
      326cb320
    • craig[bot]'s avatar
      Merge #46573 #49302 · d4115723
      craig[bot] authored
      46573: workload: fix rand generator r=jordanlewis a=jordanlewis
      
      It had rotted. Also, add JSON datatype.
      
      Closes #46569.
      
      49302: sql: enable creation of indexes on tables with user defined types r=jordanlewis a=rohany
      
      Work for #48728.
      
      This PR teaches the index backfill infrastructure to hydrate types
      before use.
      
      Release note: None
      Co-authored-by: default avatarJordan Lewis <[email protected]>
      Co-authored-by: default avatarRohan Yadav <[email protected]>
      d4115723
    • Raphael 'kena' Poss's avatar
      cliflags: fix the desc for the socket-dir flag · 2676d75d
      Raphael 'kena' Poss authored
      I had the description field mixed up when I previously
      deprecated `--socket` in favor of `--socket-dir`.
      
      Release note (bug fix): The description provided with `--help`
      on the CLI for `--socket-dir` and `--socket` has been fixed. They
      were invalid since v20.1.0.
      2676d75d
    • craig[bot]'s avatar
      Merge #49697 #49823 #49880 · b0c3eca4
      craig[bot] authored
      49697: server: use unique fake node ID in StartTenant r=asubiotto a=tbg
      
      We had previously hard-coded a NodeID of 1 (matching the underlying
      TestServer's NodeID) to make "things work" for SQL tenants. The
      commits in this lifted this restriction, so we now use a (static)
      NodeID which is highly unlikely to match any NodeID from the KV
      layer.
      
      The main work item was to make sure DistSQL does not schedule any
      flows.
      There are many places in the SQL codebase that construct flows and it
      was difficult to ensure that none of them attempt to schedule a nonlocal
      one. (We do error out when we attempt to SetupFlow them, though).
      
      Release note: None
      
      49823: adding myself to AUTHORS r=DrewKimball a=DrewKimball
      
      
      
      49880: colexec: simplify .gitignore r=yuzefovich a=yuzefovich
      
      This commit makes git ignore all files in `colexec` folder that end with
      `.eg.go` which improves the environment setup for the cases when we
      change the file names as well as simplifies the maintenance.
      
      Release note: None
      Co-authored-by: default avatarTobias Schottdorf <[email protected]>
      Co-authored-by: default avatarDrew Kimball <[email protected]>
      Co-authored-by: default avatarYahor Yuzefovich <[email protected]>
      b0c3eca4
    • craig[bot]'s avatar
      Merge #49751 #49776 · d5bd8545
      craig[bot] authored
      49751: execgen: extract template reading code r=jordanlewis a=jordanlewis
      
      Previously, all template generators had to read their template file
      themselves. Now, this is done by execgen main, opening the door to
      global transforms that affect all templates in the same way.
      
      Release note: None
      
      49776: backupccl: fix bug when backing up dropped tables with revision history r=pbardea a=pbardea
      
      When performing an incremental backup with revision history, we want to
      include all spans that were public at any point during the latest
      interval under consideration (the time between the last backup and when
      you are performing the incremental backup).
      
      However, consider a table that was dropped before the interval started.
      The table's descriptor may still be visible (in the DROPPED state). We
      should not be interested in the spans for this database. So, when going
      through the list of revisions to table descriptors, we should make sure
      that the table in question was not DROPPED at some point during this
      interval.
      
      To see why this is needed, consider the following scenario (all backups
      are assumed to be taken with revision_history):
      - Create table mydb.a
      - Create table mydb.b
      - Drop table mydb.a
      - Take a backup of mydb (full)
      - Take an incremental backup (inc) of mydb
      - Create table mydb.c
      - Take another incremental backup (inc2) of mydb
      
      The backup "inc" and "inc2" should not be considered as backing up table
      "mydb.a", since it has been dropped at that point. Note that since "inc"
      does not see any descriptor changes, only "mydb.b" is included in its
      backup. However, previously, "inc2" would see a table descriptor for
      "mydb.a" (even though it is dropped) and include it in the set of spans
      included in "inc2". This is an issue since "inc" did not include this
      span, and thus there is a gap in the coverage for this dropped table.
      
      Fixes #49707.
      
      Release note (bug fix): There was a bug where when performing
      incremental backups with revision history on a database (or full
      cluster) and a table in the database you were backing up was dropped and
      then other tables were lated create the backup would return an error.
      This is now fixed.
      Co-authored-by: default avatarJordan Lewis <[email protected]>
      Co-authored-by: default avatarPaul Bardea <[email protected]>
      d5bd8545
    • craig[bot]'s avatar
      Merge #49737 · 04445a94
      craig[bot] authored
      49737: tree: move tests that pull in server to a different package r=jordanlewis a=jordanlewis
      
      This adds the tree/eval_test package, which has dependencies on server
      and colexec. The tree_test package itself doesn't have to have these
      heavy deps anymore, which will speed up debugging cycles.
      
      Release note: None
      Co-authored-by: default avatarJordan Lewis <[email protected]>
      04445a94
    • craig[bot]'s avatar
      Merge #49822 · 6aaf3d5b
      craig[bot] authored
      49822: opt: change JoinMultiplicity from a Relational prop to a join field r=DrewKimball a=DrewKimball
      
      Previously, the JoinMultiplicity property was stored as a Relational
      prop. This is a problem because all expressions in a memo group share
      the same Relational props during exploration, and there are exploration
      rules that can flip a join's left/right multiplicity.
      
      This patch instead stores JoinMultiplcity as a join field that is
      initialized during construction of the join. This fixes the shared
      Relational props issue, and also  makes it possible for JoinMultiplicity
      to aid in calculating other logical properties.
      
      Fixes #49821
      
      Release note: None
      Co-authored-by: default avatarDrew Kimball <[email protected]>
      6aaf3d5b
    • Jordan Lewis's avatar
      execgen: extract template reading code · 6a5479be
      Jordan Lewis authored
      Previously, all template generators had to read their template file
      themselves. Now, this is done by execgen main, opening the door to
      global transforms that affect all templates in the same way.
      
      Release note: None
      6a5479be
    • Jordan Lewis's avatar
      workload: fix rand generator · 4575bdb0
      Jordan Lewis authored
      It had rotted. Also, add JSON datatype.
      
      Release note: None
      Release justification: test-only code change
      4575bdb0
    • craig[bot]'s avatar
      Merge #49855 · c51b2490
      craig[bot] authored
      49855: server,roachtest: remove leftover usage of `cockroach quit --decommission` r=irfansharif a=irfansharif
      
      Fixes #49635. This is fallout from #49350 that wasn't picked up by CI.
      
      Release note: None
      Co-authored-by: default avatarirfan sharif <[email protected]>
      c51b2490
    • Drew Kimball's avatar
      opt: change JoinMultiplicity from a Relational prop to a join field · ab412e15
      Drew Kimball authored
      Previously, the JoinMultiplicity property was stored as a Relational
      prop. This is a problem because all expressions in a memo group share
      the same Relational props during exploration, and there are exploration
      rules that can flip a join's left/right multiplicity.
      
      This patch instead stores JoinMultiplcity as a join field that is
      initialized during construction of the join. This fixes the shared
      Relational props issue, and also  makes it possible for JoinMultiplicity
      to aid in calculating other logical properties.
      
      Fixes #49821
      
      Release note: None
      ab412e15
  3. 04 Jun, 2020 13 commits
    • craig[bot]'s avatar
      Merge #49784 · 9b507562
      craig[bot] authored
      49784: config: introduce pseudo "tenants" zone r=nvanbenschoten a=nvanbenschoten
      
      Fixes #49318.
      Fixes #49445.
      Progress towards #48123.
      Informs #48774.
      
      This commit introduces a new pseudo object ID in the system tenant's namespace called "tenants". Like "liveness" and "timeseries" before it, the pseudo object allows zone configurations to be applied to pseudo-objects that do not live directly in the system tenant's SQL keyspace. In this case, the "tenants" zone allows zone configurations to be set by the system tenant and applied to all other tenants in the system. There may come a time when we want secondary tenants to have more control over their zone configurations, but that will require a much larger change to the zone config structure and UX as a whole.
      
      While making this change, we rationalize the rest of zone configuration handling and how it relates to multi-tenancy. Now that secondary tenant ranges have a zone config to call their own, we can make sense of calls from SQL into the zone configuration infrastructure. We gate off calls that don't make sense for secondary tenants and clean up hooks in SQL that handle zone config manipulation.
      
      All of this works towards a good cause - we eliminate the remaining uses of `keys.TODOSQLCodec` from `pkg/sql/...` and `pkg/config/...`, bringing us a big step closer towards being able to remove the placeholder and close #48123.
      
      This work also reveals that in order to address #48774, we need to be able to determine split points from the SystemConfig. This makes it very difficult to split on secondary tenant object (e.g. table) boundaries. However, it makes it straightforward to split on secondary tenant keysapce boundaries. This is already what we were thinking (see #47907), so out both convenience and desire, I expect that we'll follow this up with a PR that splits Ranges only at secondary tenant boundaries - placing the overhead of an otherwise empty tenant at only a single Range and a few KBs of data.
      Co-authored-by: default avatarNathan VanBenschoten <[email protected]>
      9b507562
    • craig[bot]'s avatar
      Merge #49860 · c817d37b
      craig[bot] authored
      49860: opt: add partial index predicates to TableMeta r=mgartner a=mgartner
      
      With this commit, `optbuilder` now adds partial index predicates of a
      table, as a `map[cat.IndexOrdinal]ScalarExpr`, to `TableMeta` when
      building SELECT queries. These predicates will be necessary in order to
      determine if a partial index can be used to satisfy a query.
      
      Release note: None
      Co-authored-by: default avatarMarcus Gartner <[email protected]>
      c817d37b
    • Rohan Yadav's avatar
      sql: enable creation of indexes on tables with user defined types · 2ac28f8f
      Rohan Yadav authored
      Work for #48728.
      
      This PR teaches the index backfill infrastructure to hydrate types
      before use.
      
      Release note: None
      2ac28f8f
    • craig[bot]'s avatar
      Merge #49818 · f98432f9
      craig[bot] authored
      49818: colexec: add JSONFetchVal operator for vectorized engine r=yuzefovich a=yongyanglai
      
      Previvously, the vectorized engine had no support for JSONFetchVal
      operator. This commit added JSONFetchVal operator.
      In this commit, I added BytesFamily into compatible canonical type
      family group of DatumVecCanonicalTypeFamily. Then I declared
      JSONFetchVal as supported binary operator and registered output
      type of JSONFetchVal to generate operators.
      
      Fixes #49469
      
      Release note (sql change): Vectorized engine now support JSONFetchVal(->) operator.
      Co-authored-by: default avatarYongyang Lai <[email protected]>
      f98432f9
    • Yahor Yuzefovich's avatar
      colexec: simplify .gitignore · 0b19f648
      Yahor Yuzefovich authored
      This commit makes git ignore all files in `colexec` folder that end with
      `.eg.go` which improves the environment setup for the cases when we
      change the file names as well as simplifies the maintenance.
      
      Release note: None
      0b19f648
    • Nathan VanBenschoten's avatar
      config: introduce pseudo "tenants" zone · 69531820
      Nathan VanBenschoten authored
      Fixes #49318.
      Fixes #49445.
      Progress towards #48123.
      Informs #48774.
      
      This commit introduces a new pseudo object ID in the system tenant's
      namespace called "tenants". Like "liveness" and "timeseries" before it,
      the pseudo object allows zone configurations to be applied to
      pseudo-objects that do not live directly in the system tenant's SQL
      keyspace. In this case, the "tenants" zone allows zone configurations to
      be set by the system tenant and applied to all other tenants in the
      system. There may come a time when we want secondary tenants to have
      more control over their zone configurations, but that will require a
      much larger change to the zone config structure and UX as a whole.
      
      While making this change, we rationalize the rest of zone configuration
      handling and how it relates to multi-tenancy. Now that secondary tenant
      ranges have a zone config to call their own, we can make sense of calls
      from KV into the zone configuration infrastructure. We gate off calls
      that don't make sense for secondary tenants and clean up hooks in SQL
      that handle zone config manipulation.
      
      All of this works towards a good cause - we eliminate the remaining uses
      of `keys.TODOSQLCodec` from `pkg/sql/...` and `pkg/config/...`, bringing
      us a big step closer towards being able to remove the placeholder and
      close #48123.
      
      This work also reveals that in order to address #48774, we need to be
      able to determine split points from the SystemConfig. This makes it very
      difficult to split on secondary tenant object (e.g. table) boundaries.
      However, it makes it straightforward to split on secondary tenant
      keysapce boundaries. This is already what we were thinking (see #47907),
      so out both convenience and desire, I expect that we'll follow this up
      with a PR that splits Ranges only at secondary tenant boundaries -
      placing the overhead of an otherwise empty tenant at only a single Range
      and a few KBs of data.
      69531820
    • craig[bot]'s avatar
      Merge #49836 · 588c0a3f
      craig[bot] authored
      49836: errorutil/unimplemented: use redirect server for Github links r=knz a=rafiss
      
      closes #45504
      
      This will allow us to capture telemetry such as click counts for each
      unimplemented error that is returned.
      
      Release note (general change): Links that are returned in error messages
      to point to unimplemented issues now use the CockroachLabs
      redirect/short-link server.
      Co-authored-by: default avatarRafi Shamim <[email protected]>
      588c0a3f
    • Drew Kimball's avatar
      adding myself to AUTHORS file · 89fe4be0
      Drew Kimball authored
      Release note (<category, see below>): <what> <show> <why>
      89fe4be0
    • irfan sharif's avatar
      server,roachtest: remove leftover usage of `cockroach quit --decommission` · 7e1bec52
      irfan sharif authored
      Fixes #49635. This is fallout from #49350 that wasn't picked up by CI.
      
      Release note: None
      7e1bec52
    • craig[bot]'s avatar
      Merge #49827 #49833 #49869 #49870 #49871 · 67252bc4
      craig[bot] authored
      49827: geo/geomfn: Implements ST_Segmentize for geometry r=otan a=abhishek20123g
      
      Fixes https://github.com/cockroachdb/cockroach/issues/49029
      
      This PR implements ST_Segmentize({geometry, float8}) builtin
      function, which allows modify given geometry such that no
      segment longer than the given max_segment_length.
      
      Also this PR refactors and add extra test cases for
      ST_Segmentize for geography.
      
      Release note (sql change): This PR implements ST_Segmentize({geometry,
      float8}) builtin function.
      
      49833: geo/geomfn: implement Intersection, PointOnSurface, Union r=sumeerbhola a=otan
      
      The last of the topology operators up to Chapter 20.
      
      Resolves #48951
      Resolves #49832 
      Resolves #49064
      
      Release note (sql change): Implements the ST_Intersection,
      ST_PointOnSurface and ST_Union builtin functions.
      
      49869: vendor: bump golang/protobuf to 1.4.2 r=knz a=tbg
      
      v1.4.1 aggressively deprecated something (by inserting panics) that was
      reachable via gogoproto's marshaler. Luckily, v1.4.2 has this "fixed";
      it caused enough trouble for others as well.
      
      Closes #49842.
      
      Release note: None
      
      49870: schemachange: unskip TestDropWhileBackfill r=spaskob a=spaskob
      
      Disabling the GC job was preventing this test from completing.
      Tested with `test stress`: 1000 successful runs.
      
      Fixes #44944.
      
      Release note: none.
      
      49871: kvserver: fixup test failure message r=andreimatei a=andreimatei
      
      Expected and real err were reversed.
      
      Release note: None
      Co-authored-by: default avatarabhishek20123g <[email protected]>
      Co-authored-by: default avatarOliver Tan <[email protected]>
      Co-authored-by: default avatarTobias Schottdorf <[email protected]>
      Co-authored-by: default avatarSpas Bojanov <[email protected]>
      Co-authored-by: default avatarAndrei Matei <[email protected]>
      67252bc4
    • Jordan Lewis's avatar
      tree: move tests that pull in server to a different package · 8619dd19
      Jordan Lewis authored
      This adds the tree/eval_test package, which has dependencies on server
      and colexec. The tree_test package itself doesn't have to have these
      heavy deps anymore, which will speed up debugging cycles.
      
      Release note: None
      8619dd19
    • Nathan VanBenschoten's avatar
      config: properly handle decoding errors in GetLargestObjectID · d8ae78a4
      Nathan VanBenschoten authored
      This was wrong before. Adds testing.
      d8ae78a4
    • Nathan VanBenschoten's avatar
      config: fix GetLargestObjectID for pseudo object IDs · a102b141
      Nathan VanBenschoten authored
      This was not behaving correctly, and was breaking things when a
      pseudo table had the largest ID below MaxReservedDescID.
      a102b141