简单数据分析

Installation

The library is available on JSR and NPM.

```bash

Quick setup

To quickly set up a data project with essential folders, configurations, and documentation for AI agents, you can use @nshiab/setup-data-project.

```bash

Examples

In this example, we load a CSV file with the latitude and longitude of 2023 wildfires in Canada, create point geometries from it, do a spatial join with provinces' boundaries, and then compute the number of fires and the total area burnt per province. We create charts and write the results to a file.

If you are using Deno, make sure to install and enable the Deno extension.

import { SimpleDB } from "@nshiab/simple-data-analysis";
import { barX, plot } from "@observablehq/plot";

// We start a SimpleDB instance.
const sdb = new SimpleDB();

// We create a new table
const fires = sdb.newTable("fires");
// We fetch the wildfires data. It's a csv.
await fires.loadData(
  "https://raw.githubusercontent.com/nshiab/simple-data-analysis/main/test/geodata/files/firesCanada2023.csv",
);
// We create point geometries from the lat and lon columns
// and we store the points in the new column geom
await fires.points("lat", "lon", "geom");
// We log the fires
await fires.logTable();

// We create a new table
const provinces = sdb.newTable("provinces");
// We fetch the provinces' boundaries. It's a geojson.
await provinces.loadGeoData(
  "https://raw.githubusercontent.com/nshiab/simple-data-analysis/main/test/geodata/files/CanadianProvincesAndTerritories.json",
);
// We log the provinces
await provinces.logTable();

// We match fires with provinces
// and we output the results into a new table.
// By default, joinGeo will automatically look
// for columns storing geometries in the tables,
// do a left join, and put the results
// in the left table. For non-spatial data,
// you can use the method join.
const firesInsideProvinces = await fires.joinGeo(provinces, "inside", {
  outputTable: "firesInsideProvinces",
});

// We summarize to count the number of fires
// and sum up the area burnt in each province.
await firesInsideProvinces.summarize({
  values: "hectares",
  categories: "nameEnglish",
  summaries: ["count", "sum"],
  decimals: 0,
});
// We rename columns.
await firesInsideProvinces.renameColumns({
  count: "nbFires",
  sum: "burntArea",
});
// We want the province with
// the greatest burnt area first.
await firesInsideProvinces.sort({ burntArea: "desc" });

// We log the results. By default, the method
// logs the first 10 rows, but there is 13
// rows in our data. We also log the data types.
await firesInsideProvinces.logTable({ nbRowsToLog: 13, types: true });

// We can also log a bar chart directly in the terminal...
await firesInsideProvinces.logBarChart("nameEnglish", "burntArea");

// ... or make a fancier chart or map
// with Observable Plot (don't forget to install it)
// and save it to a file.
const chart = (data: unknown[]) =>
  plot({
    marginLeft: 170,
    grid: true,
    x: { tickFormat: (d) => `${d / 1_000_000}M`, label: "Burnt area (ha)" },
    y: { label: null },
    color: { scheme: "Reds" },
    marks: [
      barX(data, {
        x: "burntArea",
        y: "nameEnglish",
        fill: "burntArea",
        sort: { y: "-x" },
      }),
    ],
  });
await firesInsideProvinces.writeChart(chart, "./chart.png");

// And we can write the data to a parquet, json or csv file.
// For geospatial data, you can use writeGeoData to
// write geojson or geoparquet files.
await firesInsideProvinces.writeData("./firesInsideProvinces.parquet");

// We close everything.
await sdb.done();

Here's what you should see in your console if your run this script.

You'll also find a chart.png file and a firesInsideProvinces.parquet file in your folder.