Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update hub.json with new unicommerce, Upscribe, Recharge, Google Analytics package in saras-daton & renamed existing packages to pascal case #250

Merged
merged 3 commits into from
Apr 5, 2023

Conversation

saras-daton
Copy link
Contributor

@saras-daton saras-daton commented Mar 21, 2023

Added unicommerce package to Saras-daton

Description

New Data Unification package for Unicommerce Daton Integration data
Renaming Existing packages to pascal case

Link to your package's repository:
https://github.com/saras-daton/AmazonAds.git
https://github.com/saras-daton/AmazonSellerCentral.git
https://github.com/saras-daton/AmazonVendorCentral.git
https://github.com/saras-daton/FacebookAds.git
https://github.com/saras-daton/Shopify.git
https://github.com/saras-daton/Unicommerce.git
https://github.com/saras-daton/Upscribe.git
https://github.com/saras-daton/Recharge.git
https://github.com/saras-daton/GoogleAnalytics.git

Checklist

This checklist is a cut down version of the best practices that we have identified as the package hub has grown. Although meeting these checklist items is not a prerequisite to being added to the Hub, we have found that packages which don't conform provide a worse user experience.

First run experience

  • The package includes a README which explains how to get started with the package and customise its behaviour
  • The README indicates which data warehouses/platforms are expected to work with this package

Customisability

  • The package uses ref or source, instead of hard-coding table references.

Packages for data transformation (delete if not relevant):

  • provide a mechanism (such as variables) to customise the location of source tables.
  • do not assume database/schema names in sources.

Dependencies

Dependencies on dbt Core

  • The package has set a supported require-dbt-version range in dbt_project.yml. Example: A package which depends on functionality added in dbt Core 1.2 should set its require-dbt-version property to [">=1.2.0", "<2.0.0"].

Dependencies on other packages defined in packages.yml:

  • Dependencies are imported from the dbt Package Hub when available, as opposed to a git installation.
  • Dependencies contain the widest possible range of supported versions, to minimise issues in dependency resolution.
  • In particular, dependencies are not pinned to a patch version unless there is a known incompatibility.

Interoperability

  • The package does not override dbt Core behaviour in such a way as to impact other dbt resources (models, tests, etc) not provided by the package.
  • The package uses the cross-database macros built into dbt Core where available, such as {{ dbt.except() }} and {{ dbt.type_string() }}.
  • The package disambiguates its resource names to avoid clashes with nodes that are likely to already exist in a project. For example, packages should not provide a model simply called users.

Versioning

  • (Required): The package's git tags validates against the regex defined in version.py
  • The package's version follows the guidance of Semantic Versioning 2.0.0. (Note in particular the recommendation for production-ready packages to be version 1.0.0 or above)

Added unicommerce package to Saras-daton
@saras-daton saras-daton changed the title Update hub.json with unicommerce package in saras-daton Update hub.json with new unicommerce package in saras-daton & rename existing packages to pascal case Mar 21, 2023
Rename existing packages to pascal case
Added new Data Unification Packages for Upscribe, Recharge, and Google Analytics
@saras-daton saras-daton changed the title Update hub.json with new unicommerce package in saras-daton & rename existing packages to pascal case Update hub.json with new unicommerce, Upscribe, Recharge, Google Analytics package in saras-daton & renamed existing packages to pascal case Apr 4, 2023
@saras-daton
Copy link
Contributor Author

Hi @joellabes @dbeatty10, Can you please review this PR and share feedback if any..

Copy link
Contributor

@joellabes joellabes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @saras-daton!

One thing I did want to mention - it seems that you're finding all your table names by querying the information schema directly. This means that none of your nodes will show up as connected to one another on the dbt DAG.

Am I correct to assume that there's something that makes it impractical to configure specific source table names to enable the full DAG to display?

@joellabes joellabes merged commit 46159a3 into dbt-labs:main Apr 5, 2023
@saras-daton
Copy link
Contributor Author

@joellabes - That is right, it is not possible to input source table name. We have to search by pattern to identify the tables and unify the data with certain deduplication and standardisation rules. All the models in a package are independent of one another. Downstream applications will source these to join and derive business KPI's.

@saras-daton
Copy link
Contributor Author

@joellabes - Need your suggestion for below:

  1. We have renamed few packages to adhere common naming standard. Example: facebook_ads is renamed to FacebookAds. However, we still see both packages appearing in dbt hub, although they point to same GIT repo. Can you help us understand the process to overcome this scenario.

Screenshot 2023-04-21 at 3 56 28 PM

  1. We also removed old releases which are no longer required from the GIT Repo, however we still see those versions in the DBT HUB Packages version dropdown

@saras-daton
Copy link
Contributor Author

@joellabes @dbeatty10 - Can you please help us with above query?

@joellabes
Copy link
Contributor

@saras-daton sorry for the delay - I've just added instructions to the hub.getdbt.com repo explaining how to do a redirect: dbt-labs/hub.getdbt.com#2481

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants