目前公司内有数据清晰+挑选的需求,新能源行业,我需要按照不同客户的需求去写python处理xlsx或csv,这个东西麻烦的地方不在python的编写,主要在需求翻译,客户传达->工程师传递->我,工程师不懂代码逻辑,我对测试了解不算很深,都是面对面碰一碰,看他要在excel里挑选什么东西,我的角色就是翻译官,我想搞个Agent之类的,但还没思路,感觉可以做成类似codex计划模式一样的效果,让ai反推工程师想要的东西,这真的是非常没有营养的东西,给这公司多多少少这4年写了小千个脚本了,我大概放一个脚本:
import pandas as pd
import file_utils
import os
files = file_utils.get_path_by_veda()
save_path = file_utils.get_save_path()
normalized_dfs = []
for f in files:
df = pd.read_parquet(f)
# --- 时间列处理 ---
if "DateTime" in df.columns:
df["DateTime"] = pd.to_datetime(df["DateTime"])
# --- 温度列处理 ---
if "CurSample5[℃]" in df.columns:
df["CurSample4[℃]"] = df["CurSample5[℃]"]
cols = ["DateTime", "CurSample1[℃]", "CurSample2[℃]", "CurSample3[℃]", "CurSample4[℃]"]
df = df[[c for c in cols if c in df.columns]].copy()
normalized_dfs.append(df)
# --- 拼接 ---
combined = pd.concat(normalized_dfs, axis=1, keys=range(len(normalized_dfs)))
# --- 结果 ---
result = pd.DataFrame()
# 1️⃣ 时间列(取第一个非空)
datetime_cols = combined.xs("DateTime", axis=1, level=1)
result["DateTime"] = datetime_cols.bfill(axis=1).iloc[:, 0]
# 2️⃣ 温度平均
for col in ["CurSample1[℃]", "CurSample2[℃]", "CurSample3[℃]", "CurSample4[℃]"]:
if col in combined.columns.get_level_values(1):
col_data = combined.xs(col, axis=1, level=1)
result[col] = col_data.mean(axis=1, skipna=True)
# 去掉空时间
result = result.dropna(subset=["DateTime"])
# 3️⃣ 排序
result = result.sort_values("DateTime").reset_index(drop=True)
# 4️⃣ 快速计算 RelativeTime
threshold = 10 # 秒
base_offset = 24 # 不需要+24就改成0
diff_sec = result["DateTime"].diff().dt.total_seconds()
is_new_segment = diff_sec.gt(threshold)
is_new_segment.iloc[0] = True
segment_id = is_new_segment.cumsum()
segment_start = result.groupby(segment_id)["DateTime"].transform("first")
segment_elapsed = (result["DateTime"] - segment_start).dt.total_seconds() / 3600
segment_max = segment_elapsed.groupby(segment_id).max()
segment_offset_map = segment_max.shift(fill_value=0).cumsum()
segment_offset = segment_id.map(segment_offset_map)
result["RelativeTime"] = base_offset + segment_elapsed + segment_offset
# 保存
result.to_parquet(os.path.join(save_path, "结果.veda"))
print("执行完毕!")