OOTB Repositories ,Export and Import scripts
@CATA
-----------
1) /atg/multisite/SiteRepository ( Export/imported as part product catalog /exportimport )
2) /atg/seo/SEORepository
3) /atg/userprofiling/PersonalizationRepository
4) /atg/commerce/catalog/ProductCatalog
5) /atg/commerce/claimable/ClaimableRepository
And any custom repositories
@PUB (6):
-------------
1) /atg/systemconfiguration/SystemConfigurationRepository
2) /atg/dynamo/security/AdminSqlRepository
3) /atg/portal/framework/PortalRepository
4) /atg/userprofiling/InternalProfileRepository
5) /atg/web/viewmapping/ViewMappingRepository
6) /atg/epub/file/PublishingFileRepository
Export Queries From CAT-A Schema :
----------------------------------
The following scripts are executed against the CATA schema.And also primary tales information is given for the validation purpose.So that we can count the data on importing also.
SEORepository :
---------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/seo/SEORepository -noTransaction -export all exported_SEO_dump.xml > exported_SEO_dump_log.txt &
Primary Table : select count(*) from das_seo_tag
ClaimableRepository:
---------------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/commerce/claimable/ClaimableRepository -noTransaction -export all exported_claimable_dump.xml > exported_claimable_dump_log.txt &
Note :
------
Need to override ClaimableRepository.properties to point the SwitchingDataSource.
Primary Table :
select count(*) from dcspp_claimable
select count(*) from dcspp_cp_folder
ProductCatalog :
----------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/commerce/catalog/ProductCatalog -noTransaction -export all exported_productCatalog_dump.xml > exported_productCatalog_dump_log.txt &
Note:Export duration will be high .It may take 13 hours .That depends on data and infrastructure.and File Size may be 7GB
PersonalizationRepository :
---------------------------
nohup startSQLRepository -m DPS -s cata_export -encoding ISO-8859-1 -repository /atg/userprofiling/PersonalizationRepository -noTransaction -export all exported_personalization_dump.xml > exported_personalization_dump_log.txt &
Note :
------
Need to override PersonalizationRepository.properties to point the SwitchingDataSource.
Primary Table :
select count(*) from dps_seg_list_folder
select count(*) from dps_seg_list
SiteRepository :
----------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/multisite/SiteRepository/ -noTransaction -export all exported_site_dump.xml > exported_site_dump_log.txt &
Primary Table :
select count(*) from site_configuration
select count(*) from site_group
Export Scripts @PUB
SystemConfigurationRepository :
-------------------------------
nohup startSQLRepository -m modules.CA -s pub_export -encoding ISO-8859-1 -repository /atg/systemconfiguration/SystemConfigurationRepository -noTransaction -export all exported_sys_conf_dump.xml > exported_sys_conf_dump_log.txt &
Primary Table :
select count(*) from das_sys_config
AdminSqlRepository :
--------------------
nohup startSQLRepository -m modules.CA -s pub_export -encoding ISO-8859-1 -repository /atg/dynamo/security/AdminSqlRepository -noTransaction -export all exported_admin_sql_dump.xml > exported_admin_sql_dump_log.txt &
Primary Table :
select count(*) from das_account
PortalRepository :
-------------------
nohup startSQLRepository -m modules.CA -s pub_export -encoding ISO-8859-1 -repository /atg/portal/framework/PortalRepository -noTransaction -export all exported_portal_dump.xml > exported_portal_dump_log.txt &
Note :
Need to modify Manifest.MF frile under CA module as per aproach document :
DafEar.Admin DCS-UI.Versioned BIZUI PubPortlet DCS-UI.SiteAdmin.Versioned SiteAdmin.Versioned DCS.Versioned DCS-UI DAF.Endeca.Index.Versioned DCS.Endeca.Index.Versioned DCS.Endeca.Index.SKUIndexing modules.PARALCommonModule modules.PARALModule modules.PARALMerchModule modules.Feed.Import modules.Feed.Export modules.Commerce
Primary Table :
select count(*) from paf_folder
select count(*) from paf_device_outputs
select count(*) from paf_display_modes
select count(*) from paf_community
select count(*) from paf_template
select count(*) from paf_page_template
select count(*) from paf_page
select count(*) from paf_layout
select count(*) from paf_style
select count(*) from paf_col_palette
select count(*) from paf_title_template
select count(*) from paf_region_def
select count(*) from paf_region
select count(*) from paf_gear_modes
select count(*) from paf_gear_def
select count(*) from paf_gear
select count(*) from paf_gear_param
select count(*) from paf_base_comm_role
select count(*) from paf_comm_template
select count(*) from paf_ct_folder
select count(*) from paf_ct_page
select count(*) from paf_ct_region
select count(*) from paf_ct_gear
select count(*) from paf_ct_alt_gear
InternalProfileRepository :
----------------------------
nohup startSQLRepository -m DPS.InternalUsers -s pub_export -encoding ISO-8859-1 -repository /atg/userprofiling/InternalProfileRepository -noTransaction -export all exported_internal_profile_dump.xml > exported_internal_profile_dump_log.txt &
Note :The following items are not export and it is also not an issue.
Not exported Items:
-------------------
workbench
workbenchTile
workbenchReport
workbenchReportItem
Primary Table :
select count(*) from dpi_user
select count(*) from dpi_contact_info
select count(*) from dpi_mailing
select count(*) from dpi_mail_batch
select count(*) from dpi_mail_server
select count(*) from dpi_role
select count(*) from dpi_access_right
select count(*) from dpi_organization
select count(*) from dpi_folder
select count(*) from wb_workbench
select count(*) from wb_tile
select count(*) from wb_report
select count(*) from wb_report_item
select count(*) from wb_report_msg
select count(*) from dsi_profile_slot
select count(*) from dsi_ind_scenario
select count(*) from dsi_coll_scenario
select count(*) from dsi_scenario_info
select count(*) from dsi_scen_mig_info
select count(*) from dsi_template_info
select count(*) from dsi_coll_trans
select count(*) from dsi_ind_trans
select count(*) from dsi_deletion
select count(*) from dsi_migration
select count(*) from dsi_xref
ViewMappingRepository :
------------------------
nohup startSQLRepository -m modules.CA -s pub_export -encoding ISO-8859-1 -repository /atg/web/viewmapping/ViewMappingRepository -noTransaction -export all exported_viewMapping_dump.xml > exported_viewMapping_dump_log.txt &
Primary Table:
select count(*) from vmap_mode
select count(*) from vmap_fh
select count(*) from vmap_im
select count(*) from vmap_ivm
select count(*) from vmap_pvm
select count(*) from vmap_iv
select count(*) from vmap_pv
select count(*) from vmap_ivattrdef
select count(*) from vmap_pvattrdef
select count(*) from vmap_attrval
Import Queries To PUB Schema :
@CATA
-----------
1) /atg/multisite/SiteRepository ( Export/imported as part product catalog /exportimport )
2) /atg/seo/SEORepository
3) /atg/userprofiling/PersonalizationRepository
4) /atg/commerce/catalog/ProductCatalog
5) /atg/commerce/claimable/ClaimableRepository
And any custom repositories
@PUB (6):
-------------
1) /atg/systemconfiguration/SystemConfigurationRepository
2) /atg/dynamo/security/AdminSqlRepository
3) /atg/portal/framework/PortalRepository
4) /atg/userprofiling/InternalProfileRepository
5) /atg/web/viewmapping/ViewMappingRepository
6) /atg/epub/file/PublishingFileRepository
Export Scripts
Export Queries From CAT-A Schema :
----------------------------------
The following scripts are executed against the CATA schema.And also primary tales information is given for the validation purpose.So that we can count the data on importing also.
SEORepository :
---------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/seo/SEORepository -noTransaction -export all exported_SEO_dump.xml > exported_SEO_dump_log.txt &
Primary Table : select count(*) from das_seo_tag
ClaimableRepository:
---------------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/commerce/claimable/ClaimableRepository -noTransaction -export all exported_claimable_dump.xml > exported_claimable_dump_log.txt &
Note :
------
Need to override ClaimableRepository.properties to point the SwitchingDataSource.
Primary Table :
select count(*) from dcspp_claimable
select count(*) from dcspp_cp_folder
ProductCatalog :
----------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/commerce/catalog/ProductCatalog -noTransaction -export all exported_productCatalog_dump.xml > exported_productCatalog_dump_log.txt &
Note:Export duration will be high .It may take 13 hours .That depends on data and infrastructure.and File Size may be 7GB
PersonalizationRepository :
---------------------------
nohup startSQLRepository -m DPS -s cata_export -encoding ISO-8859-1 -repository /atg/userprofiling/PersonalizationRepository -noTransaction -export all exported_personalization_dump.xml > exported_personalization_dump_log.txt &
Note :
------
Need to override PersonalizationRepository.properties to point the SwitchingDataSource.
Primary Table :
select count(*) from dps_seg_list_folder
select count(*) from dps_seg_list
SiteRepository :
----------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/multisite/SiteRepository/ -noTransaction -export all exported_site_dump.xml > exported_site_dump_log.txt &
Primary Table :
select count(*) from site_configuration
select count(*) from site_group
Export Scripts @PUB
SystemConfigurationRepository :
-------------------------------
nohup startSQLRepository -m modules.CA -s pub_export -encoding ISO-8859-1 -repository /atg/systemconfiguration/SystemConfigurationRepository -noTransaction -export all exported_sys_conf_dump.xml > exported_sys_conf_dump_log.txt &
Primary Table :
select count(*) from das_sys_config
AdminSqlRepository :
--------------------
nohup startSQLRepository -m modules.CA -s pub_export -encoding ISO-8859-1 -repository /atg/dynamo/security/AdminSqlRepository -noTransaction -export all exported_admin_sql_dump.xml > exported_admin_sql_dump_log.txt &
Primary Table :
select count(*) from das_account
PortalRepository :
-------------------
nohup startSQLRepository -m modules.CA -s pub_export -encoding ISO-8859-1 -repository /atg/portal/framework/PortalRepository -noTransaction -export all exported_portal_dump.xml > exported_portal_dump_log.txt &
Note :
Need to modify Manifest.MF frile under CA module as per aproach document :
DafEar.Admin DCS-UI.Versioned BIZUI PubPortlet DCS-UI.SiteAdmin.Versioned SiteAdmin.Versioned DCS.Versioned DCS-UI DAF.Endeca.Index.Versioned DCS.Endeca.Index.Versioned DCS.Endeca.Index.SKUIndexing modules.PARALCommonModule modules.PARALModule modules.PARALMerchModule modules.Feed.Import modules.Feed.Export modules.Commerce
Primary Table :
select count(*) from paf_folder
select count(*) from paf_device_outputs
select count(*) from paf_display_modes
select count(*) from paf_community
select count(*) from paf_template
select count(*) from paf_page_template
select count(*) from paf_page
select count(*) from paf_layout
select count(*) from paf_style
select count(*) from paf_col_palette
select count(*) from paf_title_template
select count(*) from paf_region_def
select count(*) from paf_region
select count(*) from paf_gear_modes
select count(*) from paf_gear_def
select count(*) from paf_gear
select count(*) from paf_gear_param
select count(*) from paf_base_comm_role
select count(*) from paf_comm_template
select count(*) from paf_ct_folder
select count(*) from paf_ct_page
select count(*) from paf_ct_region
select count(*) from paf_ct_gear
select count(*) from paf_ct_alt_gear
InternalProfileRepository :
----------------------------
nohup startSQLRepository -m DPS.InternalUsers -s pub_export -encoding ISO-8859-1 -repository /atg/userprofiling/InternalProfileRepository -noTransaction -export all exported_internal_profile_dump.xml > exported_internal_profile_dump_log.txt &
Note :The following items are not export and it is also not an issue.
Not exported Items:
-------------------
workbench
workbenchTile
workbenchReport
workbenchReportItem
Primary Table :
select count(*) from dpi_user
select count(*) from dpi_contact_info
select count(*) from dpi_mailing
select count(*) from dpi_mail_batch
select count(*) from dpi_mail_server
select count(*) from dpi_role
select count(*) from dpi_access_right
select count(*) from dpi_organization
select count(*) from dpi_folder
select count(*) from wb_workbench
select count(*) from wb_tile
select count(*) from wb_report
select count(*) from wb_report_item
select count(*) from wb_report_msg
select count(*) from dsi_profile_slot
select count(*) from dsi_ind_scenario
select count(*) from dsi_coll_scenario
select count(*) from dsi_scenario_info
select count(*) from dsi_scen_mig_info
select count(*) from dsi_template_info
select count(*) from dsi_coll_trans
select count(*) from dsi_ind_trans
select count(*) from dsi_deletion
select count(*) from dsi_migration
select count(*) from dsi_xref
ViewMappingRepository :
------------------------
nohup startSQLRepository -m modules.CA -s pub_export -encoding ISO-8859-1 -repository /atg/web/viewmapping/ViewMappingRepository -noTransaction -export all exported_viewMapping_dump.xml > exported_viewMapping_dump_log.txt &
Primary Table:
select count(*) from vmap_mode
select count(*) from vmap_fh
select count(*) from vmap_im
select count(*) from vmap_ivm
select count(*) from vmap_pvm
select count(*) from vmap_iv
select count(*) from vmap_pv
select count(*) from vmap_ivattrdef
select count(*) from vmap_pvattrdef
select count(*) from vmap_attrval
Import scripts
Import Queries To PUB Schema :
----------------------------------
InternalProfileRepository :
---------------------------
nohup startSQLRepository -m DPS.InternalUsers -s pub_import -repository /atg/userprofiling/InternalProfileRepository -import /opt/ATG/ATG10.2/home/bin/somepath/PUB/exported_internal_profile_dump.xml > imported_internal_profile_log.txt &
ProductCatalog :
------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/catalog/ProductCatalog -file /opt/ATG/ATG10.2/home/bin/exported_productCatalog_dump_withRemovedChars.xml -workspace ProductCatalogImport1 -nocheckin -batchSize 5000 > /opt/ATG/ATG10.2/home/bin/imported_productCatalog_log.txt &
[info@2020-12-03 04:28:27.614] parsing input data
CONTAINER:atg.adapter.gsa.sqlimport.ImportException; SOURCE:com.ctc.wstx.exc.WstxUnexpectedCharException: Illegal character ((CTRL-CHAR, code 26))
at [row,col {unknown-source}]: [28036853,45]
at atg.adapter.gsa.sqlimport.ImportParserImpl.parseFirsPass(ImportParserImpl.java:130)
at atg.adapter.gsa.sqlimport.SQLImporter.doImport(SQLImporter.java:1992)
at atg.adapter.gsa.sqlimport.SQLImporter.execute(SQLImporter.java:1353)
at atg.adapter.gsa.sqlimport.SQLImporter.main(SQLImporter.java:1306)
Caused by: com.ctc.wstx.exc.WstxUnexpectedCharException: Illegal character ((CTRL-CHAR, code 26))
Solution :
Note:
1)Need to add Modules.CA manifest file with below(wehre module(code base should be there in /opt/ATG/ATG10.2/) )
ATG-Required: DafEar.Admin DCS-UI.Versioned BIZUI PubPortlet DCS-UI.SiteAdmin.Versioned SiteAdmin.Versioned DCS.Versioned DCS-UI DAF.Endeca.Index.Versioned DCS.Endeca.Index.Versioned DCS.Endeca.Index.SKUIndexing --andy custom module-- modules.Commerce
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/catalog/ProductCatalog -file /opt/ATG/ATG10.2/home/bin/exported_productCatalog_dump_withRemovedChars.xml -workspace ProductCatalogImport1 -nocheckin -batchSize 5000 > /opt/ATG/ATG10.2/home/bin/imported_productCatalog_log.txt &
---------------
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/seo/SEORepository -file /opt/ATG/ATG10.2/home/bin/somePath/CorrectFiles/CAT-A/exported_SEO_dump.xml -workspace SEO_Import > imported_SEO_log.txt &
SystemConfigurationRepository :
-------------------------------
nohup startSQLRepository -m modules.CA -s pub_import -repository /atg/systemconfiguration/SystemConfigurationRepository -import /opt/ATG/ATG10.2/home/bin/somePath/CorrectFiles/PUB/exported_sys_conf_dump.xml > imported_system_conf_log.txt &
AdminSqlRepository :
--------------------
nohup startSQLRepository -m modules.CA -s pub_import -repository /atg/dynamo/security/AdminSqlRepository -import /opt/ATG/ATG10.2/home/bin/somePath/CorrectFiles/PUB/exported_admin_sql_dump.xml > imported_AdminSql_log.txt &
PortalRepository :
------------------
nohup startSQLRepository -m modules.CA -s pub_import -repository /atg/portal/framework/PortalRepository -import /opt/ATG/ATG10.2/home/bin/somePath/CorrectFiles/PUB/exported_portal_dump.xml > imported_portal_log.txt &
PersonalizationRepository :version Repository, executed first as no checkin
---------------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/userprofiling/PersonalizationRepository -file /opt/ATG/ATG10.2/home/bin/somePath/CorrectFiles/CAT-A/exported_personalization_dump.xml -workspace Personalization_Import > imported_personalization_log.txt &
ViewMappingRepository :
-----------------------
nohup startSQLRepository -m modules.CA -s pub_import -repository /atg/web/viewmapping/ViewMappingRepository -import /opt/ATG/ATG10.2/home/bin/somePath/CorrectFiles/PUB/exported_viewMapping_dump.xml > imported_viewMapping_log.txt &
ClaimableRepository :
---------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/claimable/ClaimableRepository -file /opt/ATG/ATG10.2/home/bin/exported_claimable_dump.xml -workspace Claimable_import > imported_Claimable_log.txt &
