GNIP 97: New metadata editor (#12794)

* Initial commit for the metadata architecture refactoring * improving the code * update the first handler * rename the file of the main schema * fixing manage.py * For testing a specific folder for json schemas examples was created * formatting the json schema files * update the json schema examples * adding the metadata/schema endpoint under api/v2 * rename the action of getting schema * adding the metadata/instance/{pk} endpoint * adding handlers registry * update metadata manager * update the metadata/{pk} to metadata/instance/{pk} * update the /metadata/schema endpoint * Handlers refactoring, i18n * Add TKeywords subschema * Metadata TKeywords: fix max card * TKeywords: Fix schema * Tkeywords: void get_jsonschema_instance * TKeywords: Fix autocomplete; localization * Thesaurus schema: Improve localization * TKeywords: Improve autocomplete * adding PUT functionality to the endpoint metadata/instance/{pk} * rename the view of metadata/instance/{pk} endpoint * TKeywords: Improve autocomplete * TKeywords: move tkeywords just under category field * Many improvements and addings to the base handler * Some more improvements and addings to the base handler * Return proper json schema instance * Return proper json schema instance * adding a handler for the regions field: RegionsHandler * Add DOI handler * Improvements and fixes * fixing Region autocomplete * Add DOI handler * Simplify tkeywords schema * adding serialize method to other FKs of the BaseHandler * Extending PUT and removing serialization * Fix PUT/PATCH * Fixes: now patch returns without major errors * Storing FKs to the resource model * small improvements to store FK values * TKeywords get and patch working. Added i18n to instance request * Cleanup: black and flake * Added contacts schema. Moved tkeywords autocomplete. * Load+store contacts * Added linked resources handler * Regions autocomplete * Regions load/store * Extending the Regions autocomplete results * format fixing * update the MetadataRegionsAutocomplete class * Metadata: review label i18n * Metadata: hkeywords handler - WIP * Minor improvement * Metadata: hkeywords handler * Metadata: group handler * Metadata: set owner fields as required * Metadata: doi: implement update_resource * Many improvements and fixes Sparse fields, model + handler Fix id type Handling required fields Add load_serialization_context Add null type to most optional fields Caching schema Simplified handler registration * Cleanup * Add error handling, Improve sparse field loading * Initial INSPIRE app * May improvements: sparse fields, i18n,... - Handling complex values in sparse fields - Added i18n via thesaurus - Improved subschema handling - Renamed base schema json file * tests for views * adding more tests for views * Tkeywords: hide property if no thesaurus configured * Create test errors recursively * Recurse localization in complex sparse fields * Metadata: fix contact roles * Metadata: improve handling of None values in sparse fields * Metadata: add authorization to metadata access * Metadata: fix required rolenames * Metadata: improve type handling in sparse fields * adding tests for views and manager * Metadata: improve handling of None values in sparse fields * Metadata: tentative handling of categories via autocomplete * Metadata: tentative handling of categories via autocomplete * adding base handlers tests * Metadata: tentative handling of categories via autocomplete * Metadata: handling licenses via autocomplete * Black/flake * Fix i18n caching * adding more tests for the BaseHandler * Fix flake * Black/flake * Metadata: fix group handling * Metadata: fix FK handling * adding tests for region and linkedrsources handers * fixing tests * black reformating * adding tests and reformatting * removing unused modules * removing Permissions module * adding tests for Group and Hkeyword handlers * add a flake issue * fixing views tsts * adding tests for Contact and Thesaurus handlers and autocomplete views * formatting issues * adding tests for autocomplete views and Thesaurus handler * adding tests for sparse handler * fixing format issues * Fix load_thesaurus * Remove stale sample schemas * Metadata: reload schema when labels on DB change * Fix linked resources API * Added __init__ to tests dir * Fix UserHasPerms in views * Remove geonode.inspire app * Some improvements after review * Delete unneeded migration * Some improvements after review * Metadata: show contact cardinalities only in debug mode * Fix save: make proper signals work --------- Co-authored-by: gpetrak <gkpetrak@gmail.com>
GeoNode · Jan 15, 2025 · e28ac93 · e28ac93
1 parent cb3d254
commit e28ac93
Show file tree

Hide file tree

Showing 39 changed files with 5,004 additions and 39 deletions.
diff --git a/geonode/base/admin.py b/geonode/base/admin.py
@@ -247,7 +247,7 @@ def import_rdf(self, request):
         if request.method == "POST":
             try:
                 rdf_file = request.FILES["rdf_file"]
-                name = slugify(rdf_file.name)
+                name = slugify(rdf_file.name).removesuffix("-rdf")
                 call_command("load_thesaurus", file=rdf_file, name=name)
                 self.message_user(request, "Your RDF file has been imported", messages.SUCCESS)
                 return redirect("..")

diff --git a/geonode/base/api/views.py b/geonode/base/api/views.py
@@ -1457,7 +1457,7 @@ def base_linked_resources(instance, user, params):
         return Response(data={"message": e.args[0], "success": False}, status=500, exception=True)
 
 
-def base_linked_resources_payload(instance, user, params={}):
+def base_linked_resources_instances(instance, user, params={}):
     resource_type = params.get("resource_type", None)
     link_type = params.get("link_type", None)
     type_list = resource_type.split(",") if resource_type else []
@@ -1498,7 +1498,7 @@ def base_linked_resources_payload(instance, user, params={}):
         linked_to_visib_ids = linked_to_visib.values_list("id", flat=True)
         linked_to = [lres for lres in linked_to_over_loopable if lres.target.id in linked_to_visib_ids]
 
-        ret["linked_to"] = LinkedResourceSerializer(linked_to, embed=True, many=True).data
+        ret["linked_to"] = linked_to
 
     if not link_type or link_type == "linked_by":
         linked_by_over = instance.get_linked_resources(as_target=True)
@@ -1517,11 +1517,25 @@ def base_linked_resources_payload(instance, user, params={}):
         linked_by_visib_ids = linked_by_visib.values_list("id", flat=True)
         linked_by = [lres for lres in linked_by_over_loopable if lres.source.id in linked_by_visib_ids]
 
-        ret["linked_by"] = LinkedResourceSerializer(
-            instance=linked_by, serialize_source=True, embed=True, many=True
-        ).data
+        ret["linked_by"] = linked_by
 
     if not ret["WARNINGS"]:
         ret.pop("WARNINGS")
 
     return ret
+
+
+def base_linked_resources_payload(instance, user, params={}):
+    lres = base_linked_resources_instances(instance, user, params)
+    ret = {}
+    if "linked_to" in lres:
+        ret["linked_to"] = LinkedResourceSerializer(lres["linked_to"], embed=True, many=True).data
+    if "linked_by" in lres:
+        ret["linked_by"] = LinkedResourceSerializer(
+            instance=lres["linked_by"], serialize_source=True, embed=True, many=True
+        ).data
+
+    if lres.get("WARNINGS", None):
+        ret["WARNINGS"] = lres["WARNINGS"]
+
+    return ret
diff --git a/geonode/base/management/commands/load_thesaurus.py b/geonode/base/management/commands/load_thesaurus.py
@@ -114,7 +114,8 @@ def load_thesaurus(self, input_file, name, store):
                     thesaurus_label.save()
 
         for concept in g.subjects(RDF.type, SKOS.Concept):
-            pref = preferredLabel(g, concept, default_lang)[0][1]
+            prefs = preferredLabel(g, concept, default_lang)
+            pref = prefs[0][1] if prefs else "-"
             about = str(concept)
             alt_label = g.value(concept, SKOS.altLabel, object=None, default=None)
             if alt_label is not None:

diff --git a/geonode/base/urls.py b/geonode/base/urls.py
@@ -26,7 +26,6 @@
     OwnerRightsRequestView,
     ResourceBaseAutocomplete,
     HierarchicalKeywordAutocomplete,
-    ThesaurusKeywordLabelAutocomplete,
     LinkedResourcesAutocomplete,
 )
 
@@ -57,11 +56,6 @@
         ThesaurusAvailable.as_view(),
         name="thesaurus_available",
     ),
-    re_path(
-        r"^thesaurus_autocomplete/$",
-        ThesaurusKeywordLabelAutocomplete.as_view(),
-        name="thesaurus_autocomplete",
-    ),
     re_path(
         r"^datasets_autocomplete/$",
         DatasetsAutocomplete.as_view(),

diff --git a/geonode/base/views.py b/geonode/base/views.py
@@ -334,32 +334,6 @@ class HierarchicalKeywordAutocomplete(SimpleSelect2View):
     filter_arg = "slug__icontains"
 
 
-class ThesaurusKeywordLabelAutocomplete(autocomplete.Select2QuerySetView):
-    def get_queryset(self):
-        thesaurus = settings.THESAURUS
-        tname = thesaurus["name"]
-        lang = "en"
-
-        # Filters thesaurus results based on thesaurus name and language
-        qs = ThesaurusKeywordLabel.objects.all().filter(keyword__thesaurus__identifier=tname, lang=lang)
-
-        if self.q:
-            qs = qs.filter(label__icontains=self.q)
-
-        return qs
-
-    # Overides the get results method to return custom json to frontend
-    def get_results(self, context):
-        return [
-            {
-                "id": self.get_result_value(result.keyword),
-                "text": self.get_result_label(result),
-                "selected_text": self.get_selected_result_label(result),
-            }
-            for result in context["object_list"]
-        ]
-
-
 class DatasetsAutocomplete(SimpleSelect2View):
     model = Dataset
     filter_arg = "title__icontains"

diff --git a/geonode/metadata/__init__.py b/geonode/metadata/__init__.py
diff --git a/geonode/metadata/admin.py b/geonode/metadata/admin.py
diff --git a/geonode/metadata/api/urls.py b/geonode/metadata/api/urls.py
@@ -0,0 +1,72 @@
+#########################################################################
+#
+# Copyright (C) 2020 OSGeo
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+#########################################################################
+from django.urls import path
+from rest_framework import routers
+
+from geonode.metadata.api import views
+from geonode.metadata.api.views import (
+    ProfileAutocomplete,
+    MetadataLinkedResourcesAutocomplete,
+    MetadataRegionsAutocomplete,
+    MetadataHKeywordAutocomplete,
+    MetadataGroupAutocomplete,
+)
+
+router = routers.DefaultRouter()
+router.register(r"metadata", views.MetadataViewSet, basename="metadata")
+
+urlpatterns = router.urls + [
+    path(
+        r"metadata/autocomplete/thesaurus/<thesaurusid>/keywords",
+        views.tkeywords_autocomplete,
+        name="metadata_autocomplete_tkeywords",
+    ),
+    path(r"metadata/autocomplete/users", ProfileAutocomplete.as_view(), name="metadata_autocomplete_users"),
+    path(
+        r"metadata/autocomplete/resources",
+        MetadataLinkedResourcesAutocomplete.as_view(),
+        name="metadata_autocomplete_resources",
+    ),
+    path(
+        r"metadata/autocomplete/regions",
+        MetadataRegionsAutocomplete.as_view(),
+        name="metadata_autocomplete_regions",
+    ),
+    path(
+        r"metadata/autocomplete/hkeywords",
+        MetadataHKeywordAutocomplete.as_view(),
+        name="metadata_autocomplete_hkeywords",
+    ),
+    path(
+        r"metadata/autocomplete/groups",
+        MetadataGroupAutocomplete.as_view(),
+        name="metadata_autocomplete_groups",
+    ),
+    path(
+        r"metadata/autocomplete/categories",
+        views.categories_autocomplete,
+        name="metadata_autocomplete_categories",
+    ),
+    path(
+        r"metadata/autocomplete/licenses",
+        views.licenses_autocomplete,
+        name="metadata_autocomplete_licenses",
+    ),
+    # path(r"metadata/autocomplete/users", login_required(ProfileAutocomplete.as_view()), name="metadata_autocomplete_users"),
+]