新增與重新命名欄位而不重寫檔案
閱讀約需 44 分鐘 分鐘,8 節 中共有 4 個範例。
讀完本章後,你將能在已有資料的資料表中新增欄位,在 Iceberg 中重新命名欄位而不重寫其資料檔,並能說明為什麼在 Delta Lake 中重新命名欄位等同於重寫整張資料表。
wiki.pageviews_hourly 在寫入前三個小時的資料後,收到了兩項要求。這張資料表存放 Wikimedia 頁面瀏覽量六個小時的每小時瀏覽次數,資料由第一章載入,因此每項變更都會套用在已有資料的資料表上。第一項要求是新增一個血緣欄位:每一列都應記錄是哪一次每小時載入寫入了它,以便在某次載入有誤時,能追溯到對應的資料列。資料表需要這樣的欄位,但它已有的資料列是在該欄位出現之前寫入的。第二項是名稱修正。儀表板把標題欄位稱為 article_title,而資料表稱之為 page_title,這是傾印檔所用的名稱。對於每項變更,問題都相同:在變更之前寫入的檔案會發生什麼事?
原則上,結構描述變更屬於中繼資料變更。資料檔留在原處,格式則在其中繼資料中記錄新增或重新命名的欄位。舊檔案是否仍能正確讀取,取決於格式如何在檔案內找到欄位。以名稱找出欄位的格式,在名稱改變時會遇到問題;以不會改變的依據找出欄位的格式則不會。本章說明這兩種格式各屬於哪一類、每項變更的成本為何,以及固定版本的 Delta Lake 函式庫無法進行哪一項變更。
欄位以 ID 追蹤
資料檔保存的是某些欄位的值,而資料表的綱要(schema)則說明每個值屬於哪一個欄位。要讓兩者對應起來,最簡單的做法是依名稱比對。這個做法在名稱改變之前都行得通。名稱一旦改變,綱要所指的欄位在舊檔案中就找不到同名的欄位,讀取器只能猜測舊的值究竟屬於哪個欄位。
依名稱比對的讀取器若找不到舊名稱的欄位,只有兩種不理想的選擇:讓查詢失敗,或是不聲不響地把該欄位填成 null。第二種是較隱蔽的失敗,儀表板可能顯示這種結果長達數週,才有人去檢查。我們認為悄悄出現的 null 是較嚴重的失敗,這是一種判斷,而非實測的結果。依識別碼比對可以同時避開這兩種情況,因為不論名稱為何,欄位的身分都不會變。
Iceberg 在建立欄位時會為每個欄位指定一個整數的欄位 ID(field ID),並以這個 ID 來比對欄位。名稱只是一個可以更動的標籤,Iceberg 規格也說明,重新命名欄位「必須更改名稱,但不得更改欄位 ID」[1]。讀取器開啟舊檔案時,會依欄位 ID 比對每個欄位。如果檔案中沒有某個欄位的值,規格中的投影規則會先檢查兩種在此不適用的情況,即身分分割區(identity partition)的值與名稱對應(name mapping);接著,若該欄位設有 initial-default 就回傳該值,其餘情況一律回傳 null [1]。規格為新增時帶有預設值的欄位定義了 initial-default。本章並未設定預設值,因此新增的欄位在每一筆舊資料列中都會讀到 null。
你可以自行查看這些 ID。Iceberg 資料表的綱要會列出每個欄位及其 ID,而重新命名一節的範例會印出某個欄位在重新命名前後的 ID。若想知道重新命名是否保留了欄位的身分,就該檢查這個 ID。
Delta Lake 的出發點不同。它的資料表綱要以名稱指稱每個欄位,讀取器也依名稱比對。Delta Lake 協定定義了一項選用的欄位對應(column mapping)功能,把檔案中使用的名稱與讀取者看到的名稱分開:每個欄位都有一個存放於檔案中的實體名稱,而讀取者使用的名稱是顯示名稱,可以更改而不必動到檔案。協定指出,欄位對應支援「重新命名與刪除欄位,而不必重寫所有資料」[2]。至於某個函式庫版本是否真的做到這一點,則是另一個問題。關於 Delta Lake 的一節會針對固定版本的發行版回答這個問題。
實用原則: 預期 Iceberg 的欄位在重新命名後仍保有其資料,因為讀取器是依欄位 ID 比對。預期 Delta Lake 的欄位依名稱比對,除非已啟用欄位對應。在依賴重新命名之前,先在資料表的副本上測試,並在事後讀回欄位名稱加以確認。
此部分尚未翻譯,因此以英文呈現。
Add load_batch after three hours
The first change is the one the case needs first: a column that does not exist in the files already written. In a pipeline, that happens on an ordinary day. The pipeline starts stamping each load with its number, and the table already holds rows that predate the stamp. The setup writes the first three hours, each as its own commit, so the table holds three snapshots and three data files. An Iceberg snapshot is the table's state after a commit that changes data. Each append in this chapter adds one, and a schema change adds none.
In pyiceberg 0.12.0, you add a column inside a schema update. The call takes the name and the type, and the table assigns the field ID:
with table.update_schema() as update:
update.add_column("load_batch", IntegerType())
To see what the add does to rows already on disk, commit hours 00 to 02, add the column, and append hours 03 to 05, each stamped with its hour number in load_batch. Counting rows by hour of the day splits the table at the add: hours 00 to 02 were written before it, and hours 03 to 05 after. Each hour is one write, so the split is also the split by load.
1"""Adds load_batch to a table that already holds three hours, in Iceberg and then in Delta Lake, and counts the nulls on each side of the add.2Read the four "with load_batch null" lines, which show the rows before the add and the rows after it in each format.3"""45import pyarrow as pa6from deltalake import DeltaTable, write_deltalake7from deltalake.schema import Field, PrimitiveType8from pyiceberg.types import IntegerType910from lib.lakehouse import ICEBERG_ID, delta_file_count, iceberg_file_count, new_warehouse, read_cut_hour, read_delta, split_nulls111213def stamp(rows, hour):14 """Return one hourly load with load_batch set to that load's hour number."""15 # int32 is the width IntegerType stores, so the Arrow column matches the Iceberg field16 return rows.append_column("load_batch", pa.array([hour] * rows.num_rows, type=pa.int32()))171819with new_warehouse() as wh:20 # the schema comes from hour 0, which has no load_batch yet21 table = wh.catalog.create_table(ICEBERG_ID, schema=read_cut_hour(0).schema)22 for hour in range(3): # hours 00 to 02, each one commit, all written before the add23 table.append(read_cut_hour(hour))24 print(f"iceberg snapshots before the add: {len(table.snapshots())}")25 print(f"iceberg data files before the add: {iceberg_file_count(table)}")26 with table.update_schema() as update:27 update.add_column("load_batch", IntegerType())28 print(f"iceberg snapshots after the add: {len(table.snapshots())}")29 print(f"iceberg data files after the add: {iceberg_file_count(table)}")30 for hour in range(3, 6): # hours 03 to 05, written after the add31 table.append(stamp(read_cut_hour(hour), hour))32 iceberg_rows = table.scan().to_arrow()33 print(f"iceberg rows in the table: {iceberg_rows.num_rows}")34 # the split is at 3, the first hour written after the add35 split_nulls(wh.con, iceberg_rows, "iceberg", "load_batch", 3, sides=("written before the add", "written after the add"))3637 path = f"{wh.root}/delta/pageviews_hourly"38 for hour in range(3): # hours 00 to 02, appended as three versions before the add39 write_deltalake(path, read_cut_hour(hour), mode="append")40 print(f"delta version before the add: {DeltaTable(path).version()}")41 print(f"delta data files before the add: {delta_file_count(DeltaTable(path))}")42 DeltaTable(path).alter.add_columns([Field("load_batch", PrimitiveType("integer"))])43 print(f"delta version after the add: {DeltaTable(path).version()}")44 print(f"delta data files after the add: {delta_file_count(DeltaTable(path))}")45 for hour in range(3, 6): # hours 03 to 05, written after the add46 write_deltalake(path, stamp(read_cut_hour(hour), hour), mode="append")47 delta_rows = read_delta(path)48 print(f"delta rows in the table: {delta_rows.num_rows}")49 # the same split as the Iceberg half, at 350 split_nulls(wh.con, delta_rows, "delta", "load_batch", 3, sides=("written before the add", "written after the add"))
python add-column-to-existing-files.pyiceberg snapshots before the add: 3
iceberg data files before the add: 3
iceberg snapshots after the add: 3
iceberg data files after the add: 3
iceberg rows in the table: 2999
iceberg rows written before the add: 1499
iceberg rows written before the add with load_batch null: 1499
iceberg rows written after the add: 1500
iceberg rows written after the add with load_batch null: 0
delta version before the add: 2
delta data files before the add: 3
delta version after the add: 3
delta data files after the add: 3
delta rows in the table: 2999
delta rows written before the add: 1499
delta rows written before the add with load_batch null: 1499
delta rows written after the add: 1500
delta rows written after the add with load_batch null: 0
The add changes the metadata and no data file. The snapshot count stays at 3 across the schema change, and the table keeps its 3 data files.
The 1499 rows written before the add all read null for load_batch, and 0 of the 1500 rows written after it are null. The table holds 2999 rows in all.
Delta Lake, at deltalake 1.6.6, adds a column through its alter object. The call takes a field, and the table's version goes up by one:
DeltaTable(path).alter.add_columns([Field("load_batch", PrimitiveType("integer"))])
The Delta half follows the same sequence: three early hours as three appends, the add, then hours 03 to 05. The first write is version 0, and every version number this chapter prints is Delta's real one. The version moves from 2 to 3, and the table keeps its 3 data files.
Before the add there are 1499 rows, all null for load_batch. Once the three later appends are in, the table has 2999 rows, and 0 of the 1500 rows written after the add are null. This table has no column mapping, a point the Delta Lake rename section returns to.
Both formats give the same result for this case. The old rows stay as they were written, and the column reads as null for them. The writer did not have to touch a single old file. The two halves hold the same row counts and the same null counts, so a disagreement between them would point at a reader, not at the data.
Working rule: Add a column with the format's own call, and never by rewriting the files. Expect the rows written before the add to read null for the column, and check that they do.
此部分尚未翻譯,因此以英文呈現。
Old files read as null
A null in an added column can look like missing data, and a reader who sees it may assume the old files are wrong. They are not. The old files hold the values the pipeline wrote, and the column they lack is filled on read. The rule the reader follows is the one the columns section describes: with no initial-default, no value for the field means null.
That makes the null count a useful check. Count the nulls by hour, as the add example does, and they show the pattern to expect: every row written before the add is null, and no row written after it is. Iceberg has 1499 nulls among its 1499 earlier rows, and Delta Lake has 1499 among its 1499.
If a count falls outside that pattern, a writer sent a different set of columns, or sent the column with nulls, and the cause is in the pipeline, not in the format. Look at the writer for the rows that break the pattern before you query the column, because a query over a column that is partly null will return an answer that looks reasonable.
Working rule: After an add, count nulls in the added column by hour of the day, where each hour is one write. Rows written before the add should be null, and rows written after it should not be. Any other pattern means a writer sent a different set of columns, or sent nulls in the column, so find that writer before you trust the column.
此部分尚未翻譯,因此以英文呈現。
Rename a column by its ID without rewriting files
With the field ID in place, a rename is one call in Iceberg. The call changes the name the schema reports, and keeps the ID:
with table.update_schema() as update:
update.rename_column("page_title", "article_title")
To check that the ID survives, write three hours, read the field ID of page_title, rename the column and read the ID again. The example also counts data files on both sides and queries DuckDB under the new name.
1"""Renames a column in an Iceberg table and reads the old files back under the new name.2Read the field ID lines, the data file lines and the non-null count, which show what the rename kept.3"""45import pyarrow as pa67from lib.lakehouse import (8 ICEBERG_ID,9 iceberg_file_count,10 new_warehouse,11 read_cut_hour,12 scan_arrow,13 snapshot_labels,14 sql_rows,15)1617with new_warehouse() as wh:18 table = wh.catalog.create_table(ICEBERG_ID, schema=read_cut_hour(0).schema)19 for hour in range(3):20 table.append(read_cut_hour(hour))2122 before = table.schema().find_field("page_title").field_id # the ID is what the reader matches on, not the name23 print(f"page_title field ID before the rename: {before}")24 print(f"data files before the rename: {iceberg_file_count(table)}")25 with table.update_schema() as update:26 update.rename_column("page_title", "article_title")27 print(f"article_title field ID after the rename: {table.schema().find_field('article_title').field_id}")28 print(f"data files after the rename: {iceberg_file_count(table)}")2930 rows = scan_arrow(table)31 print(f"rows: {rows.num_rows}")32 # count(column) leaves out nulls, so this counts the rows that have a value under the new name33 count = sql_rows(wh.con, "SELECT count(article_title) FROM t", t=rows)[0][0]34 print(f"article_title not null: {count}")3536 labels = snapshot_labels([snapshot.snapshot_id for snapshot in table.snapshots()])37 first = table.snapshots()[0].snapshot_id # the first snapshot, written before the rename38 print(f"{labels[first]} columns: {', '.join(scan_arrow(table, snapshot_id=first).column_names)}")
python rename-by-id.pypage_title field ID before the rename: 3
data files before the rename: 3
article_title field ID after the rename: 3
data files after the rename: 3
rows: 1499
article_title not null: 1499
snapshot 1 columns: hour_utc, domain_code, page_title, views, response_bytes
The rename keeps the field ID. page_title has ID 3 before the call, and article_title has ID 3 after it.
The data file count is 3 before the call and 3 after it, so no file was rewritten. The DuckDB query counts 1499 non-null article_title values out of 1499 rows. Every row counts, including the rows written under page_title, because the reader matches the files by ID and reads them under the new name.
The dashboards need the name, and the table does not have to be rewritten to give it to them. The data stays as it was written, and the next read sees the renamed column. The run also prints the first snapshot's columns, which are hour_utc, domain_code, page_title, views and response_bytes. A snapshot reads with the schema it was written with, so an older snapshot still reports the old name. Chapter 2 covers reading older snapshots in general, and the point here is narrower: a rename does not reach back into the history.
The cost comparison is what makes this the right call for Iceberg. A rename that keeps the ID touches the metadata and nothing else, so its cost does not grow with the size of the table. The same rename done by copying the table costs one read and one write of every row. The section on Delta Lake shows what that copy costs in that format.
Working rule: Rename an Iceberg column when its name is wrong, and check the field ID before and after the call. The field ID must be the same, and the data file count must not change. Tell every reader of the table about the rename, because the current schema has only article_title, and an older snapshot still has page_title.
在這個固定版本下,Delta Lake 沒有重新命名的功能
在這個固定版本下,Delta Lake 沒有可以做同樣重新命名的呼叫。deltalake 1.6.6 的 Delta Lake 資料表變更物件沒有重新命名的方法,其方法列在套件中 [3]。看起來像重新命名的是 set_column_metadata,因為它接受一個欄位中繼資料的字典,而且沒有任何機制阻止你傳入 name 鍵。當你傳入 name 時,這個呼叫會正常回傳,沒有錯誤。無論資料表是否啟用欄位對應(column mapping),它都不會重新命名任何東西,這就是本節要說明的陷阱。
這個陷阱先建立一個啟用欄位對應的資料表,這是協定規定的重新命名途徑,然後送出該呼叫:
DeltaTable(path).alter.set_column_metadata("page_title", {"name": "article_title"})
1"""Calls set_column_metadata with a name key on a Delta Lake table with column mapping on, then reads the names back and queries2under the new name. The columns read back line and the binder error are the lines to read.3"""45from deltalake import DeltaTable, write_deltalake67from lib.lakehouse import new_warehouse, read_cut_hour, read_delta, safe_error, sql_rows89with new_warehouse() as wh:10 path = f"{wh.root}/delta/pageviews_hourly"11 # column mapping on: the protocol's route to a rename, which this release does not offer as a call12 write_deltalake(path, read_cut_hour(0),13 configuration={"delta.columnMapping.mode": "name"})14 print(f"version before the rename: {DeltaTable(path).version()}")15 # the call raises nothing and writes a commit, so the names read back below are the only check16 DeltaTable(path).alter.set_column_metadata("page_title", {"name": "article_title"})17 print("set_column_metadata returned")18 table = DeltaTable(path)19 print(f"version after the rename: {table.version()}")20 page = next(field for field in table.schema().fields if field.name == "page_title")21 # the name key the call wrote into the field's metadata, which the reader does not use for the column name22 print(f"name in page_title metadata after the rename: {page.metadata['name']}")23 print(f"columns read back: {', '.join(read_delta(path).column_names)}")24 try:25 sql_rows(wh.con, "SELECT article_title FROM t", t=read_delta(path))26 except Exception as exc:27 print(safe_error(exc, wh))
python delta-rename-trap.pyversion before the rename: 0
set_column_metadata returned
version after the rename: 1
name in page_title metadata after the rename: article_title
columns read back: hour_utc, domain_code, page_title, views, response_bytes
BinderException: Binder Error: Referenced column "article_title" not found in FROM clause!
Candidate bindings: "page_title", "domain_code", "hour_utc", "views", "response_bytes"
LINE 1: SELECT article_title FROM t
^^^^^^^^^^^^^
呼叫正常回傳,資料表的版本從 0 變成 1,所以函式庫寫入了一次提交。name 鍵出現在該欄位的中繼資料中。但欄位仍然是 page_title:透過函式庫讀回的名稱是 hour_utc、domain_code、page_title、views 和 response_bytes。以 DuckDB 查詢 article_title 時會失敗,並出現 BinderException,錯誤訊息在可繫結的欄位中列出 page_title。
陷阱在於這個呼叫看起來像重新命名,而且執行起來也像成功的重新命名。回傳值沒有任何訊息說明名稱並未改變,只有讀取名稱才能看出實情。
在這個固定版本下,欄位對應無法完成重新命名。陷阱範例的資料表已將 delta.columnMapping.mode 設為 name,該呼叫仍然沒有改變任何名稱。協定中關於欄位對應的節描述了重新命名 [2],但 deltalake 1.6.6 沒有可執行重新命名的呼叫。
1"""Renames a Delta Lake column by overwriting the table with renamed data and schema, then reads the version before the rewrite back.2Compare the two columns lines: the version before the rewrite, and the version the rewrite makes.3"""45from deltalake import DeltaTable, write_deltalake67from lib.lakehouse import new_warehouse, read_cut_hour, read_delta89with new_warehouse() as wh:10 path = f"{wh.root}/delta/pageviews_hourly"11 for hour in range(3):12 write_deltalake(path, read_cut_hour(hour), mode="append")13 print(f"data files before the rewrite: {len(DeltaTable(path).file_uris())}")14 before = DeltaTable(path).version() # read first, so the old version can be read back after the rewrite15 print(f"version before the rewrite: {before}")16 print(f"columns before the rewrite: {', '.join(read_delta(path, version=before).column_names)}")1718 # the new names, in the table's column order: only the third one changes19 renamed = read_delta(path).rename_columns(["hour_utc", "domain_code", "article_title", "views", "response_bytes"])20 # replace the data and the schema together, in one commit, so readers of the new version see both at once21 write_deltalake(path, renamed, mode="overwrite", schema_mode="overwrite")22 print(f"data files after the rewrite: {len(DeltaTable(path).file_uris())}")23 after = DeltaTable(path).version()24 print(f"version after the rewrite: {after}")25 print(f"columns after the rewrite: {', '.join(read_delta(path).column_names)}")26 print(f"rows after the rewrite: {read_delta(path).num_rows}")
python delta-rename-by-rewrite.pydata files before the rewrite: 3
version before the rewrite: 2
columns before the rewrite: hour_utc, domain_code, page_title, views, response_bytes
data files after the rewrite: 1
version after the rewrite: 3
columns after the rewrite: hour_utc, domain_code, article_title, views, response_bytes
rows after the rewrite: 1499
在這個固定版本下,取得新名稱的方法是重寫。重寫範例透過函式庫讀取整個資料表,在記憶體中重新命名欄位,再以 mode="overwrite" 與 schema_mode="overwrite" 寫回結果,兩者會一併取代資料與結構描述。重寫前資料表有 3 個資料檔案,重寫後有 1 個。重寫產生版本 3,該版本讀到的欄位是 article_title,其中有 1499 列資料。
重寫的成本與資料表的大小成正比,因為每一列都要重新讀取並重新寫入。這裡的資料表很小,但存有數百萬列的正式環境資料表要付出相同的每列成本,所以團隊應把重寫規劃成獨立的工作。重寫之前的版本 2 讀到的仍然是 page_title,範例也會印出這一點。因此重寫只改變目前的版本,較舊的版本保留寫入時的名稱。執行重寫之前,先列出以名稱要求該欄位的讀取端。切換在一次提交中完成,所以目前版本的所有讀取端會在同一時刻一起切換,而固定在舊版本的工作則保留舊名稱。請讓這些工作與讀取端一起遷移,並在它們遷移完成之前,記錄被重寫取代的版本編號。
實用原則: 在建立資料表時就選定 Delta Lake 的欄位名稱,因為在這個固定版本下,之後的重新命名等於重寫整個資料表。不要用 set_column_metadata 重新命名欄位:它會改變版本與中繼資料,但欄位仍保留原名。任何結構描述變更之後,都要像陷阱範例那樣,透過函式庫讀回欄位名稱,並在查詢依賴這些名稱之前加以檢查。
正式環境中會出什麼問題
結構描述變更靜默失敗的情況,比出現明顯錯誤的情況更常見,而下列每一種失敗都有可以留意的徵兆。
先看沒有作用的重新命名。set_column_metadata 呼叫成功,歷史中出現一筆提交,但欄位仍保留舊名稱。這個失敗會在第一個使用 article_title 的查詢上顯現:DuckDB 查詢會因 BinderException 而失敗。執行這類查詢的排程工作,會在變更後的第一次執行就失敗,因此錯誤出現的時間點與成因很接近。
接著是歷史中殘留的舊名稱。重寫會讓目前版本使用名稱 article_title,但較早的版本仍保留寫入當時的名稱,重寫範例就顯示了這一點:版本 2 讀到的仍是 page_title。Iceberg 快照也有相同情形:第一個快照仍回報 page_title。以 article_title 讀取較早版本或快照的查詢,會找不到該欄位。依版本或快照編號讀取時,需要使用當時有效的名稱,因此固定版本編號的工作,也同時固定了它的結構描述。
第三是看起來像資料遺失的 null。新增的欄位在新增之前寫入的每一列中都會讀成 null。該欄位的 null 計數是正確的,只有在它不符合新增範例所示的前後模式時,才算是錯誤。請像新增範例那樣依小時計算 null 數量,並調查任何打破該模式的列。
最後是沒有出現在快照清單中的結構描述變更。Iceberg 的新增與重新命名,都沒有改變快照數量,檔案數量也沒有變。以計算快照數量來尋找結構描述變更的工具,會漏掉這些變更。需要知道什麼已經改變時,請依欄位 ID 與名稱直接比較結構描述本身。
實用原則: 在任何結構描述變更之後,下游查詢執行之前,請做三件事:透過函式庫讀回欄位名稱;依小時計算每個新增欄位的 null 數量,其中每個小時代表一次寫入;並將結構描述(而非快照數量)與該變更原本要達成的結果相比對。
本章的決定
本節的表格將兩種格式並列,對照本章所談的各項變更。每個儲存格都出自本章的某個範例。
| 變更 | Iceberg | Delta Lake |
|---|---|---|
| 新增欄位 | 可行。舊資料列讀出來為 null,快照數量維持在 3 | 可行。舊資料列讀出來為 null,版本號加一 |
| 重新命名欄位 | 可行。欄位 ID 不變,也不會重寫任何資料檔 | 沒有重新命名的呼叫。set_column_metadata 會提交,但不會更動任何名稱;重寫之後才會得到新名稱 |
| 讀取較舊的快照或版本 | 顯示舊名稱 | 顯示舊名稱 |
- 在兩種格式中,都可以對已有資料的資料表新增欄位。舊資料列讀取該欄位時為 null,新增範例中依小時統計的數量也確認了這個模式。我們建議採用這種做法,而不要複製整張資料表,因為它不會寫入任何舊檔案。
- 以
rename_column重新命名 Iceberg 欄位,並檢查其欄位 ID 在前後是否相同。凡是名稱有誤的 Iceberg 欄位,我們都建議這樣做,因為其成本與資料表的大小無關。 - 在建立資料表時就選定 Delta Lake 的欄位名稱。在固定的版本下,於 Delta Lake 重新命名等於重寫整張資料表,名稱有誤時就必須針對這項成本做好規劃。
- 不要用
set_column_metadata重新命名 Delta Lake 欄位。該呼叫會成功,但完全不會更動名稱。 - 在這個固定版本下,不要指望以欄位對應(column mapping)來重新命名。函式庫沒有可執行重新命名的呼叫,因此開啟對應功能並不會帶來重新命名的能力。
- 每次變更結構描述之後,在查詢依賴這些欄位名稱之前,都要在兩種格式中讀回欄位名稱。
此部分尚未翻譯,因此以英文呈現。
Sources
- Apache Iceberg Table Spec, Schema Evolution (Column Projection), https://iceberg.apache.org/spec/, read Nov 6, 2026. Quoted: "Renaming an existing field must change the name, but not the field ID." The null rule, which the text paraphrases, is in the same section.
- Delta Transaction Log Protocol, Column Mapping section, tag v4.4.1, https://raw.githubusercontent.com/delta-io/delta/v4.4.1/PROTOCOL.md, read Nov 6, 2026. Quoted: "the renaming and dropping of columns without having to rewrite all the data."
- deltalake 1.6.6, the methods of
DeltaTable.alterin the installed package:add_columns,add_constraint,add_feature,drop_column_not_null,drop_constraint,set_column_metadata,set_table_description,set_table_nameandset_table_properties. None renames a column. To list them, rundir(DeltaTable(path).alter).
本章,於本版次中
版次 2026.11.10範例於 2026年11 月 20 通過測試本章 4 個範例 個範例中有 4 個通過
譯自英文書,版次 2026.11.10。