Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON schema and yaml support #1068

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

EmilyGraceSeville7cf
Copy link

@EmilyGraceSeville7cf EmilyGraceSeville7cf commented Aug 13, 2023

Related Issue/Addition to code

  • Addition to code.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Proposed Changes

JSON schema generator and converter to .ini file.

  • Syntax to generate JSON schema: nu generate_json_schema.nu --schema <where-to-put-it> or ./generate_json_schema.sh --schema <where-to-put-it>
  • Syntax to generate INI from YAML: nu generate_json_schema.nu --ini-config <where-to-put-ini> --yaml-config <where-to-take-yaml-from> or ./generate_json_schema.sh --ini-config <where-to-put-ini> --yaml-config <where-to-take-yaml-from>

Why is this change needed?

It's needed because there is no intellisence available for .ini files.

Additional Info

This script allows to:

  • generate JSON schema for YAML config (from remote default INI config in this repo)
  • rewrite INI config from YAML config (not to write INI directly)

It means that users can configure tool via YAML having intellisence and validation. Script is written in Nushell and is already ported to POSIX sh.

Currently this schema is automatically generated:

{
    "$schema": "http://json-schema.org/draft-07/schema",
    "title": "config",
    "description": "A config",
    "type": "object",
    "properties": {
      "config-meta": {
        "title": "config-meta",
        "description": "config-meta\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
        "type": "object",
        "properties": {
          "use_this_config": {
            "title": "use this config",
            "description": "use this config\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": [
              "string",
              "boolean"
            ],
            "default": "Ask"
          },
          "this_config_description": {
            "title": "this config description",
            "description": "this config description\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "My Custom Config"
          },
          "configs_path": {
            "title": "configs path",
            "description": "configs path\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "configs"
          }
        },
        "additionalProperties": false
      },
      "general": {
        "title": "general",
        "description": "general\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
        "type": "object",
        "properties": {
          "your_channel_id": {
            "title": "your channel id",
            "description": "your channel id\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "Ask"
          },
          "auto_check_update": {
            "title": "auto check update",
            "description": "auto check update\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "boolean",
            "default": true
          },
          "release_channel": {
            "title": "release channel",
            "description": "release channel\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "All"
          },
          "skip_confirm_video": {
            "title": "skip confirm video",
            "description": "skip confirm video\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "boolean",
            "default": false
          },
          "moderator_mode": {
            "title": "moderator mode",
            "description": "moderator mode\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "boolean",
            "default": false
          },
          "auto_close": {
            "title": "auto close",
            "description": "auto close\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "boolean",
            "default": false
          },
          "colors_enabled": {
            "title": "colors enabled",
            "description": "colors enabled\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "boolean",
            "default": true
          }
        },
        "additionalProperties": false
      },
      "scanning": {
        "title": "scanning",
        "description": "scanning\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
        "type": "object",
        "properties": {
          "scan_mode": {
            "title": "scan mode",
            "description": "scan mode\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "Ask"
          },
          "max_comments": {
            "title": "max comments",
            "description": "max comments\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": [
              "string",
              "integer"
            ],
            "default": "Ask"
          },
          "videos_to_scan": {
            "title": "videos to scan",
            "description": "videos to scan\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "Ask"
          },
          "channel_to_scan": {
            "title": "channel to scan",
            "description": "channel to scan\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "Ask"
          },
          "recent_videos_amount": {
            "title": "recent videos amount",
            "description": "recent videos amount\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": [
              "string",
              "integer"
            ],
            "minimum": 1,
            "maximum": 5000,
            "default": "Ask"
          }
        },
        "additionalProperties": false
      },
      "filtering": {
        "title": "filtering",
        "description": "filtering\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
        "type": "object",
        "properties": {
          "filter_mode": {
            "title": "filter mode",
            "description": "filter mode\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "Ask"
          },
          "filter_subMode": {
            "title": "filter submode",
            "description": "filter submode\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "Ask"
          },
          "channel_ids_to_filter": {
            "title": "channel ids to filter",
            "description": "channel ids to filter\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "Ask"
          },
          "autoASCII_sensitivity": {
            "title": "autoascii sensitivity",
            "description": "autoascii sensitivity\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": [
              "string",
              "integer"
            ],
            "minimum": 1,
            "maximum": 3,
            "default": "Ask"
          },
          "characters_to_filter": {
            "title": "characters to filter",
            "description": "characters to filter\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "Ask"
          },
          "strings_to_filter": {
            "title": "strings to filter",
            "description": "strings to filter\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "Ask"
          },
          "regex_to_filter": {
            "title": "regex to filter",
            "description": "regex to filter\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "Ask"
          }
        },
        "additionalProperties": false
      },
      "autosmart-customization": {
        "title": "autosmart-customization",
        "description": "autosmart-customization\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
        "type": "object",
        "properties": {
          "detect_link_spam": {
            "title": "detect link spam",
            "description": "detect link spam\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "boolean",
            "default": true
          },
          "detect_sub_challenge_spam": {
            "title": "detect sub challenge spam",
            "description": "detect sub challenge spam\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "boolean",
            "default": false
          },
          "detect_spam_threads": {
            "title": "detect spam threads",
            "description": "detect spam threads\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "boolean",
            "default": true
          }
        },
        "additionalProperties": false
      },
      "advanced-filtering": {
        "title": "advanced-filtering",
        "description": "advanced-filtering\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
        "type": "object",
        "properties": {
          "duplicate_check_modes": {
            "title": "duplicate check modes",
            "description": "duplicate check modes\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "AutoSmart, SensitiveSmart"
          },
          "minimum_duplicates": {
            "title": "minimum duplicates",
            "description": "minimum duplicates\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "integer",
            "minimum": 3,
            "default": 8
          },
          "minimum_duplicate_length": {
            "title": "minimum duplicate length",
            "description": "minimum duplicate length\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "integer",
            "minimum": 1,
            "default": 14
          },
          "levenshtein_distance": {
            "title": "levenshtein distance",
            "description": "levenshtein distance\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "number",
            "minimum": 0,
            "maximum": 1,
            "default": 0.9
          },
          "stolen_comments_check_modes": {
            "title": "stolen comments check modes",
            "description": "stolen comments check modes\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "SensitiveSmart"
          },
          "stolen_minimum_text_length": {
            "title": "stolen minimum text length",
            "description": "stolen minimum text length\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "integer",
            "minimum": 1,
            "default": 25
          },
          "fuzzy_stolen_comment_detection": {
            "title": "fuzzy stolen comment detection",
            "description": "fuzzy stolen comment detection\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "boolean",
            "default": false
          }
        },
        "additionalProperties": false
      },
      "removal": {
        "title": "removal",
        "description": "removal\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
        "type": "object",
        "properties": {
          "skip_deletion": {
            "title": "skip deletion",
            "description": "skip deletion\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "boolean",
            "default": false
          },
          "delete_without_reviewing": {
            "title": "delete without reviewing",
            "description": "delete without reviewing\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "boolean",
            "default": false
          },
          "enable_ban": {
            "title": "enable ban",
            "description": "enable ban\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": [
              "string",
              "boolean"
            ],
            "default": "Ask"
          },
          "remove_all_author_comments": {
            "title": "remove all author comments",
            "description": "remove all author comments\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": [
              "string",
              "boolean"
            ],
            "default": "Ask"
          },
          "removal_type": {
            "title": "removal type",
            "description": "removal type\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "rejected"
          },
          "whitelist_excluded": {
            "title": "whitelist excluded",
            "description": "whitelist excluded\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": [
              "string",
              "boolean"
            ],
            "default": "Ask"
          },
          "check_deletion_success": {
            "title": "check deletion success",
            "description": "check deletion success\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "boolean",
            "default": true
          }
        },
        "additionalProperties": false
      },
      "logging": {
        "title": "logging",
        "description": "logging\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
        "type": "object",
        "properties": {
          "enable_logging": {
            "title": "enable logging",
            "description": "enable logging\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": [
              "string",
              "boolean"
            ],
            "default": "Ask"
          },
          "log_path": {
            "title": "log path",
            "description": "log path\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "logs"
          },
          "log_mode": {
            "title": "log mode",
            "description": "log mode\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "plaintext"
          },
          "json_log": {
            "title": "json log",
            "description": "json log\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "boolean",
            "default": false
          },
          "json_encoding": {
            "title": "json encoding",
            "description": "json encoding\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "string",
            "default": "utf-8"
          },
          "json_extra_data": {
            "title": "json extra data",
            "description": "json extra data\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "boolean",
            "default": true
          },
          "json_log_all_comments": {
            "title": "json log all comments",
            "description": "json log all comments\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "boolean",
            "default": false
          },
          "json_profile_picture": {
            "title": "json profile picture",
            "description": "json profile picture\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": [
              "string",
              "boolean"
            ],
            "default": false
          }
        },
        "additionalProperties": false
      },
      "api": {
        "title": "api",
        "description": "api\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
        "type": "object",
        "properties": {
          "quota_limit": {
            "title": "quota limit",
            "description": "quota limit\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "integer",
            "default": 10000
          }
        },
        "additionalProperties": false
      },
      "info": {
        "title": "info",
        "description": "info\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
        "type": "object",
        "properties": {
          "config_version": {
            "title": "config version",
            "description": "config version\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini",
            "type": "integer",
            "default": 32
          }
        },
        "additionalProperties": false
      }
    }
  }

Checklist:

  • My code follows the style guidelines of this project and I have read CONTRIBUTING.md
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published in downstream modules
  • I have checked my code and corrected any misspellings

Screenshots

image

@EmilyGraceSeville7cf EmilyGraceSeville7cf marked this pull request as draft August 13, 2023 15:36
@EmilyGraceSeville7cf
Copy link
Author

Ideally, YAML should be supported by this tool itself. But as I am not Python developer, I can't integrate YAML support directly to Python code now. (Maybe later.)

@EmilyGraceSeville7cf EmilyGraceSeville7cf marked this pull request as ready for review August 13, 2023 17:09
@EmilyGraceSeville7cf
Copy link
Author

EmilyGraceSeville7cf commented Aug 17, 2023

Now it's possible to tinker ./.vscode/settings.json (just this path is checked for existence) automatically when --setup-vscode option is passed with --yaml-config <path> --schema <path> to enable intellisence for generated JSON schema in one step.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant