INNER CODE UNIT · Python

objs

HIT-SCIR/Chinese-Mixtral-8x7B · data/parquet2jsonl.py:32

            objs = parse_file(root / file)
            for obj_str in list(objs):
                try:
                    obj = json.loads(obj_str)
                    f.write(f"{json.dumps(obj, ensure_ascii=False)}\n")
                except:
                    print("json parse error")


if __name__ == "__main__":
    fire.Fire(main)

View source record →