Smaller files: don't serialize turn geometry?

[dabreegster]

Not sure this will work, but it's a good little exploratory project. map_model/src/turn.rs stores geom: PolyLine. Last time I used the serialized_size_bytes stuff in map.rs, these polylines constituted a huge part of the binary map files. The release size is slowly growing, so it'd be nice to trim this down.

Maybe we could skip serializing the PolyLine, then in Map::new after deserializing, go fill it in again. Should measure the impact on file sizes and initial map loading time.

[JavedNissar]

@dabreegster I'm a bit confused here. geom: PolyLine seems like it's used quite a bit so wouldn't skipping serialization of geom entail removing geom? As we derive the Serialize trait from the struct, we can't really change how it serializes things right?

[dabreegster]

The idea is to apply #[serde(skip_serializing, skip_deserializing)] to geom: PolyLine in Turn. Then in Map::new (which should be the only entrypoint for loading a serialized map), go through and fill out the geometry again. Could consider doing it lazily with a RefCell if the initial load time becomes higher and it's the case that most turn geometry isn't needed upfront.

I haven't looked at this in a while, so before starting, it's also worth uncommenting the code in Map::new and double-checking that turn geometry indeed takes up lots of the space.

[JavedNissar]

pub fn new(path: String, timer: &mut Timer) -> Map {
        if path.starts_with(&abstutil::path_all_maps()) {
            match abstutil::maybe_read_binary(path.clone(), timer) {
                Ok(map) => {
                    let map: Map = map;

                    if false {
                        use abstutil::{prettyprint_usize, serialized_size_bytes};
                        println!(
                            "- {} roads: {} bytes",
                            prettyprint_usize(map.roads.len()),
                            prettyprint_usize(serialized_size_bytes(&map.roads))
                        );
                        println!(
                            "- {} lanes: {} bytes",
                            prettyprint_usize(map.lanes.len()),
                            prettyprint_usize(serialized_size_bytes(&map.lanes))
                        );
                        println!(
                            "- {} intersections: {} bytes",
                            prettyprint_usize(map.intersections.len()),
                            prettyprint_usize(serialized_size_bytes(&map.intersections))
                        );
                        println!(
                            "- {} turns: {} bytes",
                            prettyprint_usize(map.turns.len()),
                            prettyprint_usize(serialized_size_bytes(&map.turns))
                        );
                        println!(
                            "- {} buildings: {} bytes",
                            prettyprint_usize(map.buildings.len()),
                            prettyprint_usize(serialized_size_bytes(&map.buildings))
                        );
                        println!(
                            "- {} areas: {} bytes",
                            prettyprint_usize(map.areas.len()),
                            prettyprint_usize(serialized_size_bytes(&map.areas))
                        );
                        println!(
                            "- {} parking lots: {} bytes",
                            prettyprint_usize(map.parking_lots.len()),
                            prettyprint_usize(serialized_size_bytes(&map.parking_lots))
                        );
                        println!(
                            "- {} zones: {} bytes",
                            prettyprint_usize(map.zones.len()),
                            prettyprint_usize(serialized_size_bytes(&map.zones))
                        );
                        // This is the partridge in the pear tree, I suppose
                        println!(
                            "- pathfinder: {} bytes",
                            prettyprint_usize(serialized_size_bytes(&map.pathfinder))
                        );
                    }

                    return map;
                }
                Err(err) => {
                    Map::corrupt_err(path, err);
                    std::process::exit(1);
                }
            }
        }

        let raw: RawMap = if path.starts_with(&abstutil::path_all_raw_maps()) {
            abstutil::read_binary(path, timer)
        } else {
            // Synthetic
            abstutil::read_json(path, timer)
        };
        Map::create_from_raw(raw, true, timer)
    }

Where is the commented out code in Map::new?

[dabreegster]

Er sorry, the if false bit

[dabreegster]

I poked at this a little and learned

1. Actually, buildings are taking lots of the file size right now, because of the polygons. Few fixes incoming to squeeze it down a bit, but there's probably lots to gain from doing some more efficient serialization of all the Pt2D's
2. For vehicle turns, it should be easy to make the code from make/turns.rs apply for a Turn later to fill in the geometry lazily. But for walking turns, it might be more complicated to figure out which case of the geom to generate. Still possible, but maybe a bit annoying.