apunkt/trustgraph
1
0
Fork 0
mirror of https://github.com/trustgraph-ai/trustgraph.git synced 2026-05-02 11:52:37 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

Feature/pkgsplit (#83)

Browse source 
* Starting to spawn base package
* More package hacking
* Bedrock and VertexAI
* Parquet split
* Updated templates
* Utils
This commit is contained in:
cybermaggedon 2024-09-30 19:36:09 +01:00 • committed by GitHub
parent 3fb75c617b
commit 9b91d5eee3
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
262 changed files with 630 additions and 420 deletions
Download patch file Download diff file Expand all files Collapse all files

24
trustgraph-core/scripts/dump-parquet
View file

@ -1,24 +0,0 @@
#!/usr/bin/env python3
import pyarrow as pa
import pyarrow.csv as pc
import pyarrow.parquet as pq
import pandas as pd
import sys
df = None
for file in sys.argv[1:]:
part = pq.read_table(file).to_pandas()
if df is None:
df = part
else:
df = pd.concat([df, part], ignore_index=True)
if df is not None:
table = pa.Table.from_pandas(df)
pc.write_csv(table, sys.stdout.buffer)