ATG BCC cleaning orphan assets and cleaning Versioned data
As there are many orphan assets (products and SKU) and unwanted versioned content(purging not there earlier ), we got the task from client to remove the all unwanted orphan skus/proudct
This we achieved in the following way.We called this as BCC baseline.
1)We took the the catA/CatB, then idenity the all orphan Assets( skus and prodyucts) and then removed at catA/catB through DB Query.
2)Identify GWP promotion SKU's & Product's for the current environment for excluding in the Orphan product sku.
3)Then create projects at BCC by adding all these orphand asstes(skus and proudcts in multiple projects), then execute the project(after business team approve).
1)We need to communite with team regarding the freeze preiod for BCC.
2)Need to turn off the outbound feed.
3)Business team should stop using BCC.
4)Export the BCC topology
5)Shunt down the BCC instance.
6)Backup the versionFileStore file folder from BCC instance and delete the orginal versionFileStore file folder from BCC instance of PROD instance
7)Export the data dump of PUB schema and CATA schema of PROD oracle DB
8)Import the data dump of PUB and CATA schmea to SIT 2 DB(PUB ==> BASELINE_EXP_BCC_PROD & CATA ==> BASELINE_EXP_CATA_PROD)
9)Create Two Empty schema with specific schema meta data only[DDL only] one for PUB as "BASELINE_IMP_BCC_PROD", one for CATA as "BASELINE_IMP_CATA_PROD" in the same SIT 2 DB
10)Export the data dump for only price and inventory tables like dcs_plfol_chld, dcs_child_fol_pl, dcs_gen_fol_pl, dcs_price_level, dcs_price_levels,dcs_complex_price,dcs_price,dcs_price_list,dcs_inventory,dcs_inv_atp from PROD core
11)Import the data dump of price and inventory tables like dcs_plfol_chld, dcs_child_fol_pl, dcs_gen_fol_pl, dcs_price_level, dcs_price_levels,dcs_complex_price,dcs_price,dcs_price_list,dcs_inventory,dcs_inv_atp into "BASELINE_ATGCORE" schema of SIT-2 DB
12)Update the "DISTRIBUTED_LOCK_TIMEOUT" value to 3600 seconds in SIT-2 DB and restart the SIT-2 DB instance for reflecting the updated value. This modification is required for full deployment purpose.
13)Audit the count of records for all user tables from "BASELINE_EXP_BCC_PROD" and "BASELINE_EXP_CATA_PROD" schema
14)Create an XLS and store the count of records from both schema's in the expected format(We will see later this format)
15)Export the insert table script for das_id_generator table from "BASELINE_EXP_BCC_PROD" schema
16) Make sure all BCC instance tables exists in the "BASELINE_IMP_BCC_PROD" schema after the DBA create new schema with only meta data.
17) Make sure all BCC instance tables exists in the "BASELINE_IMP_CATA_PROD" schema after the DBA create new schema with only meta data..
18)Execute the grant access SQL script in "BASELINE_IMP_CATA_PROD" schema to grant DML access of required tables to "BASELINE_EXP_BCC_PROD" schema
19)Execute the disable constraints SQL script in "BASELINE_IMP_CATA_PROD" schema
20)Import the insert table script for das_id_generator table data into "BASELINE_IMP_BCC_PROD" schema
Execute the Export/Import Process
1)Configure the BASELINE_EXP_BCC_PROD & BASELINE_EXP_CATA_PROD schema's in Build Box using the "MonitoredDataSource" & "FakeXADataSource" classes in the "cata_export" & "pub_export" server local config layer
2)Configure the BASELINE_IMP_BCC_PROD & BASELINE_IMP_CATA_PROD schema's in Build Box using the "MonitoredDataSource" & "FakeXADataSource" classes in the "pub_import" server local config layer
3)Configure the CORE schema of the SIT 2 DB in "cata_export", "pub_export" & "pub_import" server local config layer
4)Execute the Repository Export process as per the Approach/Release document using "cata_export" & "pub_export" server local config layer
5)Verify the exported Files
6)Execute the Import Process for all the repositories as per the Approach/Release document using "pub_import" server local config layer
7)Execute the product catalog manual check-in process using the "ProductCatalog_Manually_checkin.sql" script
8)Audit the count of records for all user tables from the newly imported data in "BASELINE_IMP_BCC_PROD" and "BASELINE_IMP_CATA_PROD" schema
9)Store the new count of records from both "BASELINE_IMP_BCC_PROD" and "BASELINE_IMP_CATA_PROD" schema's in the expected format. Shared to client for review purpose.
10)Create/update the datasource for "BASELINE_IMP_BCC_PROD" and "BASELINE_IMP_CATA_PROD" schema's in SIT weblogic console
11)Shut down the BCC instances of SIT2
12)Convert the existing BCC instance SwitchingDataSource_production and SwitchingDataSource_staging components from switching type to normal datasource by changing the class to atg.nucleus.JNDIReference and adding JNDI name through ATG-Data layer
13)Add the RL related hot fix in the BCC instance EAR and also add/update the RL related component changes in the BCC server local config layer
14)Start only SIT BCC instance and this instance must point to the datasource of "BASELINE_IMP_BCC_PROD" and "BASELINE_IMP_CATA_PROD" schema's
15)Execute the RL through SIT-BCC instance dyn/admin
16)Validate the newly generated versionFileStore location and publishingFileRepository
2)Convert the existing e-Store instance SwitchingDataSource component from switching type to normal datasource by changing the class name and adding JNDI name through ATG-Data layer. It will point to "BASELINE_IMP_CATA_PROD"
3)Change the password of the datasource, if applicable
4)Make sure the weblogic datasources of SIT 2 BCC instance and e-store instance point to new imported schemas.
5)Bring Up the BCC server
6)Verify the BCC logs
7)Bring up the store and Aux servers
8)Verify the Store Aux Logs
9)Verify the eStore Instance
10)Configure the BCC agents
11)Perform basic validation of BCC deployment and also publish empty project
12)Perform BCC Full Deployment
13)Verify the BCC publishing of catalog, content etc repositories assets after modification
14)verify the BCC deployment of config file system assets like targeters, scenarios, user segement etc by either creating the new asset or updating the exist asset
15)Execute the enable constraints SQL script in "BASELINE_IMP_CATA_PROD" schema
16)Execute CMS in one of the agent
17)Perform Endeca Indexing in SIT-2 environment
18)Verify the Estore instance and mainly the browse and shop pages
Production CutOver
1)Copy the versionFile store to a predefined location
2)Export the data dump from newly imported "BASELINE_IMP_BCC_PROD" and "BASELINE_IMP_CATA_PROD" schema from SIT2
3)Import the exported data dump from "BASELINE_IMP_BCC_PROD" and "BASELINE_IMP_CATA_PROD" schema of SIT2 schema into QA DB as PUB2,CATA2,CATB2,PRVCATA2,PRVCATB2
4)Delete the data from DMS related tables, E-PUB related tables, DSS server ID tables
5)Bring down all the instances of PROD
6)Make the weblogic data source changes to point to the new schemas as above
7)Bring up the store,aux instances etc..
8)Bring up the BCC instances
9)Verify the logs
10)Import the BCC Topology
11)Perform Sample Deployment
12)Perform Stibo Deployment
13)Perform Indexing
14)Verify the Functionality/site
15)Users Testing
Export scripts/commands (from DB to XML file)
The export scripts are executed via repository.It will generate the xml files and log txt files.
As part of the export commands, we have to provide the output file and log file names also.
And at each exort command level, the count query is mentioned for the primary table to validate the data it self.
CAT-A (10):
-----------
1) /atg/multisite/SiteRepository
2) /atg/seo/SEORepository
3) /atg/userprofiling/PersonalizationRepository
4) /atg/commerce/catalog/ProductCatalog
5) /atg/commerce/claimable/ClaimableRepository
6) /atg/commerce/locations/LocationRepository
7) /com/xx/giftregistry/repo/GiftRegistryRepository
8) /com/xx/newsLetter/NewsSubscriptionRepository
9) /com/xx/repo/BrandNumbersRepository
10) /com/xx/common/content/statictext/repo/ContentRepository
From PUB (6):
-------------
1) /atg/systemconfiguration/SystemConfigurationRepository
2) /atg/dynamo/security/AdminSqlRepository
3) /atg/portal/framework/PortalRepository
4) /atg/userprofiling/InternalProfileRepository
5) /atg/web/viewmapping/ViewMappingRepository
6) /atg/epub/file/PublishingFileRepository
LocationRepository :
--------------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/commerce/locations/LocationRepository -noTransaction -export all exported_location_dump.xml > exported_location_dump_log.txt &
Primary Table :
select count(*) from dcs_location
SEORepository :
---------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/seo/SEORepository -noTransaction -export all exported_SEO_dump.xml > exported_SEO_dump_log.txt &
Primary Table :
select count(*) from das_seo_tag
ClaimableRepository:
---------------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/commerce/claimable/ClaimableRepository -noTransaction -export all exported_claimable_dump.xml > exported_claimable_dump_log.txt &
Note :
------
Need to override ClaimableRepository.properties to point the SwitchingDataSource.
Primary Table :
select count(*) from dcspp_claimable
select count(*) from dcspp_cp_folder
ProductCatalog :
----------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/commerce/catalog/ProductCatalog -noTransaction -export all exported_productCatalog_dump.xml > exported_productCatalog_dump_log.txt &
Duration : 13 hours 15 mins
File Size : 7136205408 MB(7.136205408 GB)
GiftRegistryRepository :
------------------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /com/xx/giftregistry/repo/GiftRegistryRepository -noTransaction -export all exported_giftregistry_dump.xml > exported_giftregistry_dump_log.txt &
Primary Table :
select count(*) from xx_event
NewsSubscriptionRepository :
-----------------------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /com/xx/newsLetter/NewsSubscriptionRepository -noTransaction -export all exported_newsSubscription_dump.xml > exported_newsSubscription_dump_log.txt &
Primary Table :
select count(*) from xx_NEWS_LETTER
BrandNumbersRepository :
-------------------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /com/xx/repo/BrandNumbersRepository -noTransaction -export all exported_brandnumbers_dump.xml > exported_brandnumbers_dump_log.txt &
Primary Table :
select count(*) from xx_BRAND_NUMBERS
ContentRepository :
--------------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /com/xx/common/content/statictext/repo/ContentRepository -noTransaction -export all exported_content_dump.xml > exported_content_dump_log.txt &
Primary Table :
select count(*) from xx_static_text 4964
select count(*) from xx_page 258
select count(*) from xx_site_big_static_text 611
select count(*) from xx_site_label_static_text 16896
select count(*) from xx_site_msg_static_text 2044
PersonalizationRepository :
---------------------------
nohup startSQLRepository -m DPS -s cata_export -encoding ISO-8859-1 -repository /atg/userprofiling/PersonalizationRepository -noTransaction -export all exported_personalization_dump.xml > exported_personalization_dump_log.txt &
Note :
------
Need to override PersonalizationRepository.properties to point the SwitchingDataSource.
Primary Table :
select count(*) from dps_seg_list_folder
select count(*) from dps_seg_list
SiteRepository :
----------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/multisite/SiteRepository/ -noTransaction -export all exported_site_dump.xml > exported_site_dump_log.txt &
Primary Table :
select count(*) from site_configuration
select count(*) from site_group
SystemConfigurationRepository :
-------------------------------
nohup startSQLRepository -m modules.CA -s pub_export -encoding ISO-8859-1 -repository /atg/systemconfiguration/SystemConfigurationRepository -noTransaction -export all exported_sys_conf_dump.xml > exported_sys_conf_dump_log.txt &
Primary Table :
select count(*) from das_sys_config
AdminSqlRepository :
--------------------
nohup startSQLRepository -m modules.CA -s pub_export -encoding ISO-8859-1 -repository /atg/dynamo/security/AdminSqlRepository -noTransaction -export all exported_admin_sql_dump.xml > exported_admin_sql_dump_log.txt &
Primary Table :
select count(*) from das_account
PortalRepository :
-------------------
nohup startSQLRepository -m modules.CA -s pub_export -encoding ISO-8859-1 -repository /atg/portal/framework/PortalRepository -noTransaction -export all exported_portal_dump.xml > exported_portal_dump_log.txt &
Note :
Need to modify Manifest.MF file under CA module :
DafEar.Admin DCS-UI.Versioned BIZUI PubPortlet DCS-UI.SiteAdmin.Versioned SiteAdmin.Versioned DCS.Versioned DCS-UI DAF.Endeca.Index.Versioned DCS.Endeca.Index.Versioned DCS.Endeca.Index.SKUIndexing modules.PARALCommonModule modules.PARALModule modules.PARALMerchModule modules.Feed.Import modules.Feed.Export modules.Commerce
Primary Table :
select count(*) from paf_folder
select count(*) from paf_device_outputs
select count(*) from paf_display_modes
select count(*) from paf_community
select count(*) from paf_template
select count(*) from paf_page_template
select count(*) from paf_page
select count(*) from paf_layout
select count(*) from paf_style
select count(*) from paf_col_palette
select count(*) from paf_title_template
select count(*) from paf_region_def
select count(*) from paf_region
select count(*) from paf_gear_modes
select count(*) from paf_gear_def
select count(*) from paf_gear
select count(*) from paf_gear_param
select count(*) from paf_base_comm_role
select count(*) from paf_comm_template
select count(*) from paf_ct_folder
select count(*) from paf_ct_page
select count(*) from paf_ct_region
select count(*) from paf_ct_gear
select count(*) from paf_ct_alt_gear
InternalProfileRepository :
----------------------------
nohup startSQLRepository -m DPS.InternalUsers -s pub_export -encoding ISO-8859-1 -repository /atg/userprofiling/InternalProfileRepository -noTransaction -export all exported_internal_profile_dump.xml > exported_internal_profile_dump_log.txt &
Note :
Not exported Items:
-------------------
workbench
workbenchTile
workbenchReport
workbenchReportItem
Primary Table :
select count(*) from dpi_user
select count(*) from dpi_contact_info
select count(*) from dpi_mailing
select count(*) from dpi_mail_batch
select count(*) from dpi_mail_server
select count(*) from dpi_role
select count(*) from dpi_access_right
select count(*) from dpi_organization
select count(*) from dpi_folder
select count(*) from wb_workbench
select count(*) from wb_tile
select count(*) from wb_report
select count(*) from wb_report_item
select count(*) from wb_report_msg
select count(*) from dsi_profile_slot
select count(*) from dsi_ind_scenario
select count(*) from dsi_coll_scenario
select count(*) from dsi_scenario_info
select count(*) from dsi_scen_mig_info
select count(*) from dsi_template_info
select count(*) from dsi_coll_trans
select count(*) from dsi_ind_trans
select count(*) from dsi_deletion
select count(*) from dsi_migration
select count(*) from dsi_xref
ViewMappingRepository :
------------------------
nohup startSQLRepository -m modules.CA -s pub_export -encoding ISO-8859-1 -repository /atg/web/viewmapping/ViewMappingRepository -noTransaction -export all exported_viewMapping_dump.xml > exported_viewMapping_dump_log.txt &
Primary Table:
select count(*) from vmap_mode
select count(*) from vmap_fh
select count(*) from vmap_im
select count(*) from vmap_ivm
select count(*) from vmap_pvm
select count(*) from vmap_iv
select count(*) from vmap_pv
select count(*) from vmap_ivattrdef
select count(*) from vmap_pvattrdef
select count(*) from vmap_attrval
CAT-A (10):
-----------
1) /atg/multisite/SiteRepository (imported as part product catalog import )
2) /atg/seo/SEORepository
3) /atg/userprofiling/PersonalizationRepository
4) /atg/commerce/catalog/ProductCatalog
5) /atg/commerce/claimable/ClaimableRepository
6) /atg/commerce/locations/LocationRepository
7) /com/xx/giftregistry/repo/GiftRegistryRepository
8) /com/xx/newsLetter/NewsSubscriptionRepository
9) /com/xx/repo/BrandNumbersRepository
10) /com/xx/common/content/statictext/repo/ContentRepository
From PUB (6):
-------------
1) /atg/systemconfiguration/SystemConfigurationRepository
2) /atg/dynamo/security/AdminSqlRepository
3) /atg/portal/framework/PortalRepository
4) /atg/userprofiling/InternalProfileRepository
5) /atg/web/viewmapping/ViewMappingRepository
6) /atg/epub/file/PublishingFileRepository
Import Queries To PUB Schema :
----------------------------------
InternalProfileRepository :
---------------------------
nohup startSQLRepository -m DPS.InternalUsers -s pub_import -repository /atg/userprofiling/InternalProfileRepository -import /opt/ATG/ATG10.2/home/bin/Exported_Files_On_1_20_2017/CorrectFiles/PUB/exported_internal_profile_dump.xml > imported_internal_profile_log.txt &
ProductCatalog :
------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/catalog/ProductCatalog -file /opt/ATG/ATG10.2/home/bin/Exported_Files_On_1_20_2017/CorrectFiles/CAT-A/exported_productCatalog_dump.xml -workspace ProductCatalogImport -nocheckin > imported_productCatalog_log.txt &
[info@2020-12-03 04:28:27.608] created new workspace: workspace:75800
[info@2020-12-03 04:28:27.614] parsing input data
CONTAINER:atg.adapter.gsa.sqlimport.ImportException; SOURCE:com.ctc.wstx.exc.WstxUnexpectedCharException: Illegal character ((CTRL-CHAR, code 26))
at [row,col {unknown-source}]: [28036853,45]
at atg.adapter.gsa.sqlimport.ImportParserImpl.parseFirsPass(ImportParserImpl.java:130)
at atg.adapter.gsa.sqlimport.SQLImporter.doImport(SQLImporter.java:1992)
at atg.adapter.gsa.sqlimport.SQLImporter.execute(SQLImporter.java:1353)
at atg.adapter.gsa.sqlimport.SQLImporter.main(SQLImporter.java:1306)
Caused by: com.ctc.wstx.exc.WstxUnexpectedCharException: Illegal character ((CTRL-CHAR, code 26))
at [row,col {unknown-source}]: [28036853,45]
at com.ctc.wstx.sr.StreamScanner.throwInvalidSpace(StreamScanner.java:666)
at com.ctc.wstx.sr.StreamScanner.throwInvalidSpace(StreamScanner.java:651)
at com.ctc.wstx.sr.BasicStreamReader.readCDataPrimary(BasicStreamReader.java:4226)
at com.ctc.wstx.sr.BasicStreamReader.nextFromTreeCommentOrCData(BasicStreamReader.java:3285)
at com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2801)
at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1065)
at org.codehaus.stax2.ri.Stax2EventReaderImpl.nextEvent(Stax2EventReaderImpl.java:255)
at atg.adapter.gsa.sqlimport.ImportParserImpl.parseFirsPass(ImportParserImpl.java:112)
... 3 more
Solution :
sed -ie 's/few/asd/g' hello.txt
Run below on increasing maxConnection @JTData component at server layer and also
Need to add Modules.CA manifest file with below
ATG-Required: DafEar.Admin DCS-UI.Versioned BIZUI PubPortlet DCS-UI.SiteAdmin.Versioned SiteAdmin.Versioned DCS.Versioned DCS-UI DAF.Endeca.Index.Versioned DCS.Endeca.Index.Versioned DCS.Endeca.Index.SKUIndexing modules.PARALCommonModule modules.PARALModule modules.PARALMerchModule modules.Feed.Import modules.Feed.Export modules.Commerce
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/catalog/ProductCatalog -file /opt/ATG/ATG10.2/home/bin/exported_productCatalog_dump_22_1_2017_withRemovedChars.xml -workspace ProductCatalogImport -nocheckin -batchSize 5000 > /opt/ATG/ATG10.2/home/bin/imported_productCatalog_log_siva1.txt &
@163 box
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/catalog/ProductCatalog -file /opt/ATG/ATG10.2/home/bin/Apparao/exported_productCatalog_dump_22_1_2017_withRemovedChars.xml -workspace ProductCatalogImport1 -nocheckin -batchSize 5000 > /opt/ATG/ATG10.2/home/bin/imported_productCatalog_log_apparao_24_1.txt &
workspace :76800
LocationRepository :
---------------------
./startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/locations/LocationRepository -file exported_location_dump.xml -workspace location_import_workspace
**** Error Thu Dec 03 04:23:13 IST 2020 1606949593882 / Unable to set configured property "/com/xx/feed/export/listener/IncrementalPriceItemIndexingListener.incrementalQueueRepository" atg.nucleus.ConfigurationException: Unable to resolve component /atg/search/repository/IncrementalItemQueueRepository
**** Error Thu Dec 03 04:23:13 IST 2020 1606949593915 / Unable to set configured property "/atg/search/SynchronizationInvoker.port" atg.nucleus.ConfigurationException: Unable to resolve component /atg/search/Constants.null
**** Error Thu Dec 03 04:23:13 IST 2020 1606949593916 / Unable to set configured property "/atg/search/SynchronizationInvoker.host" atg.nucleus.ConfigurationException: Unable to resolve component /atg/search/Constants.null
**** info Thu Dec 03 04:23:13 IST 2020 1606949593967 /atg/epub/deployment/DeploymentManager Resolving reference to /atg/deployment/DeploymentManager
[info@2020-12-03 04:23:14.089] created new workspace: workspace:75701
[info@2020-12-03 04:23:14.096] parsing input data
[info@2020-12-03 04:23:14.569] importing 184 items.
[info@2020-12-03 04:23:14.569] checking out versioned items in batch
[info@2020-12-03 04:23:15.599] performing add/update/removes
[info@2020-12-03 04:23:16.825] Phase 2 of 2: 100%
[info@2020-12-03 04:23:16.826] checkIn: workspace:75701
java.lang.AbstractMethodError: atg.adapter.version.VersionRepository.getVersionItemsInLine(Latg/versionmanager/DevelopmentLine;ZZZ)Ljava/util/Iterator;
at atg.adapter.secure.GenericSecuredMutableVersionRepository.getVersionItemsInLine(GenericSecuredMutableVersionRepository.java:883)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.get(DevelopmentLineRepositoryImpl.java:547)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.getAllAssetVersions(DevelopmentLineRepositoryImpl.java:376)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.getAllAssetVersions(DevelopmentLineRepositoryImpl.java:393)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.getAllAssetVersions(DevelopmentLineRepositoryImpl.java:351)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.getAllAssetVersionsCount(DevelopmentLineRepositoryImpl.java:404)
at atg.versionmanager.impl.WorkspaceRepositoryImpl.checkInAll(WorkspaceRepositoryImpl.java:1490)
at atg.adapter.gsa.sqlimport.SQLImporter.checkIn(SQLImporter.java:2512)
at atg.adapter.gsa.sqlimport.SQLImporter.doImport(SQLImporter.java:2043)
at atg.adapter.gsa.sqlimport.SQLImporter.execute(SQLImporter.java:1353)
at atg.adapter.gsa.sqlimport.SQLImporter.main(SQLImporter.java:1306)
Solution: Again done @ 163 box. So issue.
only media-extranal item port :
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/catalog/ProductCatalog -file /opt/ATG/ATG10.2/home/bin/exported_media_external_dump.xml -workspace Only_MediaItem_Import -nocheckin > Only_MediaItem_Import.txt &
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/catalog/ProductCatalog -file /opt/ATG/ATG10.2/home/bin/exported_media1_external_dump.xml -workspace exported_media1_external_dump -nocheckin > exported_media1_external_dump.txt &
SEORepository :
---------------
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/seo/SEORepository -file /opt/ATG/ATG10.2/home/bin/Exported_Files_On_1_20_2017/CorrectFiles/CAT-A/exported_SEO_dump.xml -workspace SEO_Import > imported_SEO_log.txt &
log:
[info@2020-12-05 04:46:48.034] parsing input data
[info@2020-12-05 04:46:48.323] importing 239 items.
[info@2020-12-05 04:46:48.323] checking out versioned items in batch
[info@2020-12-05 04:46:49.338] performing add/update/removes
[info@2020-12-05 04:46:50.472] Phase 2 of 2: 100%
[info@2020-12-05 04:46:50.472] checkIn: workspace:76100
java.lang.AbstractMethodError: atg.adapter.version.VersionRepository.getVersionItemsInLine(Latg/versionmanager/DevelopmentLine;ZZZ)Ljava/util/Iterator;
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.get(DevelopmentLineRepositoryImpl.java:547)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.getAllAssetVersions(DevelopmentLineRepositoryImpl.java:376)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.getAllAssetVersions(DevelopmentLineRepositoryImpl.java:393)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.getAllAssetVersions(DevelopmentLineRepositoryImpl.java:351)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.getAllAssetVersionsCount(DevelopmentLineRepositoryImpl.java:404)
at atg.versionmanager.impl.WorkspaceRepositoryImpl.checkInAll(WorkspaceRepositoryImpl.java:1490)
at atg.adapter.gsa.sqlimport.SQLImporter.checkIn(SQLImporter.java:2512)
at atg.adapter.gsa.sqlimport.SQLImporter.doImport(SQLImporter.java:2043)
at atg.adapter.gsa.sqlimport.SQLImporter.execute(SQLImporter.java:1353)
at atg.adapter.gsa.sqlimport.SQLImporter.main(SQLImporter.java:1306)
Solution: Again run at 163 box. Assets are checked in.
SystemConfigurationRepository :
-------------------------------
nohup startSQLRepository -m modules.CA -s pub_import -repository /atg/systemconfiguration/SystemConfigurationRepository -import /opt/ATG/ATG10.2/home/bin/Exported_Files_On_1_20_2017/CorrectFiles/PUB/exported_sys_conf_dump.xml > imported_system_conf_log.txt &
AdminSqlRepository :
--------------------
nohup startSQLRepository -m modules.CA -s pub_import -repository /atg/dynamo/security/AdminSqlRepository -import /opt/ATG/ATG10.2/home/bin/Exported_Files_On_1_20_2017/CorrectFiles/PUB/exported_admin_sql_dump.xml > imported_AdminSql_log.txt &
PortalRepository :
------------------
nohup startSQLRepository -m modules.CA -s pub_import -repository /atg/portal/framework/PortalRepository -import /opt/ATG/ATG10.2/home/bin/Exported_Files_On_1_20_2017/CorrectFiles/PUB/exported_portal_dump.xml > imported_portal_log.txt &
PersonalizationRepository :version Repository, executed first as no checkin
---------------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/userprofiling/PersonalizationRepository -file /opt/ATG/ATG10.2/home/bin/Exported_Files_On_1_20_2017/CorrectFiles/CAT-A/exported_personalization_dump.xml -workspace Personalization_Import > imported_personalization_log.txt &
ViewMappingRepository :
-----------------------
nohup startSQLRepository -m modules.CA -s pub_import -repository /atg/web/viewmapping/ViewMappingRepository -import /opt/ATG/ATG10.2/home/bin/Exported_Files_On_1_20_2017/CorrectFiles/PUB/exported_viewMapping_dump.xml > imported_viewMapping_log.txt &
ClaimableRepository :
---------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/claimable/ClaimableRepository -file /opt/ATG/ATG10.2/home/bin/Apparao/exported_claimable_dump.xml -workspace Claimable_import > imported_Claimable_log_apparao.txt &
LocationRepository : agin run at 163 box
---------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/locations/LocationRepository -file /opt/ATG/ATG10.2/home/bin/Apparao/exported_location_dump.xml -workspace location_import_workspace > imported_location_log_apparao.txt &
GiftRegistryRepository :
------------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /com/xx/giftregistry/repo/GiftRegistryRepository -file /opt/ATG/ATG10.2/home/bin/Apparao/exported_giftregistry_dump.xml -workspace GiftRegistry_import > imported_giftRegistry_log_apparao.txt &
NewsSubscriptionRepository :
-----------------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /com/xx/newsLetter/NewsSubscriptionRepository -file /opt/ATG/ATG10.2/home/bin/Apparao/exported_newsSubscription_dump.xml -workspace NewsSubscription_import > imported_newsSubscription_log_apparao.txt &
BrandNumbersRepository :
------------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /com/xx/repo/BrandNumbersRepository -file /opt/ATG/ATG10.2/home/bin/Apparao/exported_brandnumbers_dump.xml -workspace BrandNumbers_imported > imported_brandnumber_log_apparao.txt &
ContentRepository :
-------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /com/xx/common/content/statictext/repo/ContentRepository -file /opt/ATG/ATG10.2/home/bin/Apparao/exported_content_dump.xml -workspace Content_imported > imported_content_log_apparao.txt &
As there are many orphan assets (products and SKU) and unwanted versioned content(purging not there earlier ), we got the task from client to remove the all unwanted orphan skus/proudct
This we achieved in the following way.We called this as BCC baseline.
1)We took the the catA/CatB, then idenity the all orphan Assets( skus and prodyucts) and then removed at catA/catB through DB Query.
2)Identify GWP promotion SKU's & Product's for the current environment for excluding in the Orphan product sku.
3)Then create projects at BCC by adding all these orphand asstes(skus and proudcts in multiple projects), then execute the project(after business team approve).
Now Export and Import DB.
While doing,we have to follow the below steps carefullly.1)We need to communite with team regarding the freeze preiod for BCC.
2)Need to turn off the outbound feed.
3)Business team should stop using BCC.
4)Export the BCC topology
5)Shunt down the BCC instance.
6)Backup the versionFileStore file folder from BCC instance and delete the orginal versionFileStore file folder from BCC instance of PROD instance
7)Export the data dump of PUB schema and CATA schema of PROD oracle DB
8)Import the data dump of PUB and CATA schmea to SIT 2 DB(PUB ==> BASELINE_EXP_BCC_PROD & CATA ==> BASELINE_EXP_CATA_PROD)
9)Create Two Empty schema with specific schema meta data only[DDL only] one for PUB as "BASELINE_IMP_BCC_PROD", one for CATA as "BASELINE_IMP_CATA_PROD" in the same SIT 2 DB
10)Export the data dump for only price and inventory tables like dcs_plfol_chld, dcs_child_fol_pl, dcs_gen_fol_pl, dcs_price_level, dcs_price_levels,dcs_complex_price,dcs_price,dcs_price_list,dcs_inventory,dcs_inv_atp from PROD core
11)Import the data dump of price and inventory tables like dcs_plfol_chld, dcs_child_fol_pl, dcs_gen_fol_pl, dcs_price_level, dcs_price_levels,dcs_complex_price,dcs_price,dcs_price_list,dcs_inventory,dcs_inv_atp into "BASELINE_ATGCORE" schema of SIT-2 DB
12)Update the "DISTRIBUTED_LOCK_TIMEOUT" value to 3600 seconds in SIT-2 DB and restart the SIT-2 DB instance for reflecting the updated value. This modification is required for full deployment purpose.
13)Audit the count of records for all user tables from "BASELINE_EXP_BCC_PROD" and "BASELINE_EXP_CATA_PROD" schema
14)Create an XLS and store the count of records from both schema's in the expected format(We will see later this format)
15)Export the insert table script for das_id_generator table from "BASELINE_EXP_BCC_PROD" schema
16) Make sure all BCC instance tables exists in the "BASELINE_IMP_BCC_PROD" schema after the DBA create new schema with only meta data.
17) Make sure all BCC instance tables exists in the "BASELINE_IMP_CATA_PROD" schema after the DBA create new schema with only meta data..
18)Execute the grant access SQL script in "BASELINE_IMP_CATA_PROD" schema to grant DML access of required tables to "BASELINE_EXP_BCC_PROD" schema
19)Execute the disable constraints SQL script in "BASELINE_IMP_CATA_PROD" schema
20)Import the insert table script for das_id_generator table data into "BASELINE_IMP_BCC_PROD" schema
Execute the Export/Import Process
1)Configure the BASELINE_EXP_BCC_PROD & BASELINE_EXP_CATA_PROD schema's in Build Box using the "MonitoredDataSource" & "FakeXADataSource" classes in the "cata_export" & "pub_export" server local config layer
2)Configure the BASELINE_IMP_BCC_PROD & BASELINE_IMP_CATA_PROD schema's in Build Box using the "MonitoredDataSource" & "FakeXADataSource" classes in the "pub_import" server local config layer
3)Configure the CORE schema of the SIT 2 DB in "cata_export", "pub_export" & "pub_import" server local config layer
4)Execute the Repository Export process as per the Approach/Release document using "cata_export" & "pub_export" server local config layer
5)Verify the exported Files
6)Execute the Import Process for all the repositories as per the Approach/Release document using "pub_import" server local config layer
7)Execute the product catalog manual check-in process using the "ProductCatalog_Manually_checkin.sql" script
8)Audit the count of records for all user tables from the newly imported data in "BASELINE_IMP_BCC_PROD" and "BASELINE_IMP_CATA_PROD" schema
9)Store the new count of records from both "BASELINE_IMP_BCC_PROD" and "BASELINE_IMP_CATA_PROD" schema's in the expected format. Shared to client for review purpose.
10)Create/update the datasource for "BASELINE_IMP_BCC_PROD" and "BASELINE_IMP_CATA_PROD" schema's in SIT weblogic console
11)Shut down the BCC instances of SIT2
12)Convert the existing BCC instance SwitchingDataSource_production and SwitchingDataSource_staging components from switching type to normal datasource by changing the class to atg.nucleus.JNDIReference and adding JNDI name through ATG-Data layer
13)Add the RL related hot fix in the BCC instance EAR and also add/update the RL related component changes in the BCC server local config layer
14)Start only SIT BCC instance and this instance must point to the datasource of "BASELINE_IMP_BCC_PROD" and "BASELINE_IMP_CATA_PROD" schema's
15)Execute the RL through SIT-BCC instance dyn/admin
16)Validate the newly generated versionFileStore location and publishingFileRepository
Verify the DB Import in SIT 2
1)Shut down the instances of SIT2(eStore,BCC,Aux)2)Convert the existing e-Store instance SwitchingDataSource component from switching type to normal datasource by changing the class name and adding JNDI name through ATG-Data layer. It will point to "BASELINE_IMP_CATA_PROD"
3)Change the password of the datasource, if applicable
4)Make sure the weblogic datasources of SIT 2 BCC instance and e-store instance point to new imported schemas.
5)Bring Up the BCC server
6)Verify the BCC logs
7)Bring up the store and Aux servers
8)Verify the Store Aux Logs
9)Verify the eStore Instance
10)Configure the BCC agents
11)Perform basic validation of BCC deployment and also publish empty project
12)Perform BCC Full Deployment
13)Verify the BCC publishing of catalog, content etc repositories assets after modification
14)verify the BCC deployment of config file system assets like targeters, scenarios, user segement etc by either creating the new asset or updating the exist asset
15)Execute the enable constraints SQL script in "BASELINE_IMP_CATA_PROD" schema
16)Execute CMS in one of the agent
17)Perform Endeca Indexing in SIT-2 environment
18)Verify the Estore instance and mainly the browse and shop pages
Production CutOver
1)Copy the versionFile store to a predefined location
2)Export the data dump from newly imported "BASELINE_IMP_BCC_PROD" and "BASELINE_IMP_CATA_PROD" schema from SIT2
3)Import the exported data dump from "BASELINE_IMP_BCC_PROD" and "BASELINE_IMP_CATA_PROD" schema of SIT2 schema into QA DB as PUB2,CATA2,CATB2,PRVCATA2,PRVCATB2
4)Delete the data from DMS related tables, E-PUB related tables, DSS server ID tables
5)Bring down all the instances of PROD
6)Make the weblogic data source changes to point to the new schemas as above
7)Bring up the store,aux instances etc..
8)Bring up the BCC instances
9)Verify the logs
10)Import the BCC Topology
11)Perform Sample Deployment
12)Perform Stibo Deployment
13)Perform Indexing
14)Verify the Functionality/site
15)Users Testing
Export scripts/commands (from DB to XML file)
The export scripts are executed via repository.It will generate the xml files and log txt files.
As part of the export commands, we have to provide the output file and log file names also.
And at each exort command level, the count query is mentioned for the primary table to validate the data it self.
CAT-A (10):
-----------
1) /atg/multisite/SiteRepository
2) /atg/seo/SEORepository
3) /atg/userprofiling/PersonalizationRepository
4) /atg/commerce/catalog/ProductCatalog
5) /atg/commerce/claimable/ClaimableRepository
6) /atg/commerce/locations/LocationRepository
7) /com/xx/giftregistry/repo/GiftRegistryRepository
8) /com/xx/newsLetter/NewsSubscriptionRepository
9) /com/xx/repo/BrandNumbersRepository
10) /com/xx/common/content/statictext/repo/ContentRepository
From PUB (6):
-------------
1) /atg/systemconfiguration/SystemConfigurationRepository
2) /atg/dynamo/security/AdminSqlRepository
3) /atg/portal/framework/PortalRepository
4) /atg/userprofiling/InternalProfileRepository
5) /atg/web/viewmapping/ViewMappingRepository
6) /atg/epub/file/PublishingFileRepository
--------------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/commerce/locations/LocationRepository -noTransaction -export all exported_location_dump.xml > exported_location_dump_log.txt &
Primary Table :
select count(*) from dcs_location
SEORepository :
---------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/seo/SEORepository -noTransaction -export all exported_SEO_dump.xml > exported_SEO_dump_log.txt &
Primary Table :
select count(*) from das_seo_tag
ClaimableRepository:
---------------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/commerce/claimable/ClaimableRepository -noTransaction -export all exported_claimable_dump.xml > exported_claimable_dump_log.txt &
Note :
------
Need to override ClaimableRepository.properties to point the SwitchingDataSource.
Primary Table :
select count(*) from dcspp_claimable
select count(*) from dcspp_cp_folder
ProductCatalog :
----------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/commerce/catalog/ProductCatalog -noTransaction -export all exported_productCatalog_dump.xml > exported_productCatalog_dump_log.txt &
Duration : 13 hours 15 mins
File Size : 7136205408 MB(7.136205408 GB)
GiftRegistryRepository :
------------------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /com/xx/giftregistry/repo/GiftRegistryRepository -noTransaction -export all exported_giftregistry_dump.xml > exported_giftregistry_dump_log.txt &
Primary Table :
select count(*) from xx_event
NewsSubscriptionRepository :
-----------------------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /com/xx/newsLetter/NewsSubscriptionRepository -noTransaction -export all exported_newsSubscription_dump.xml > exported_newsSubscription_dump_log.txt &
Primary Table :
select count(*) from xx_NEWS_LETTER
BrandNumbersRepository :
-------------------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /com/xx/repo/BrandNumbersRepository -noTransaction -export all exported_brandnumbers_dump.xml > exported_brandnumbers_dump_log.txt &
Primary Table :
select count(*) from xx_BRAND_NUMBERS
ContentRepository :
--------------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /com/xx/common/content/statictext/repo/ContentRepository -noTransaction -export all exported_content_dump.xml > exported_content_dump_log.txt &
Primary Table :
select count(*) from xx_static_text 4964
select count(*) from xx_page 258
select count(*) from xx_site_big_static_text 611
select count(*) from xx_site_label_static_text 16896
select count(*) from xx_site_msg_static_text 2044
PersonalizationRepository :
---------------------------
nohup startSQLRepository -m DPS -s cata_export -encoding ISO-8859-1 -repository /atg/userprofiling/PersonalizationRepository -noTransaction -export all exported_personalization_dump.xml > exported_personalization_dump_log.txt &
Note :
------
Need to override PersonalizationRepository.properties to point the SwitchingDataSource.
Primary Table :
select count(*) from dps_seg_list_folder
select count(*) from dps_seg_list
SiteRepository :
----------------
nohup startSQLRepository -m modules.Commerce -s cata_export -encoding ISO-8859-1 -repository /atg/multisite/SiteRepository/ -noTransaction -export all exported_site_dump.xml > exported_site_dump_log.txt &
Primary Table :
select count(*) from site_configuration
select count(*) from site_group
SystemConfigurationRepository :
-------------------------------
nohup startSQLRepository -m modules.CA -s pub_export -encoding ISO-8859-1 -repository /atg/systemconfiguration/SystemConfigurationRepository -noTransaction -export all exported_sys_conf_dump.xml > exported_sys_conf_dump_log.txt &
Primary Table :
select count(*) from das_sys_config
AdminSqlRepository :
--------------------
nohup startSQLRepository -m modules.CA -s pub_export -encoding ISO-8859-1 -repository /atg/dynamo/security/AdminSqlRepository -noTransaction -export all exported_admin_sql_dump.xml > exported_admin_sql_dump_log.txt &
Primary Table :
select count(*) from das_account
PortalRepository :
-------------------
nohup startSQLRepository -m modules.CA -s pub_export -encoding ISO-8859-1 -repository /atg/portal/framework/PortalRepository -noTransaction -export all exported_portal_dump.xml > exported_portal_dump_log.txt &
Note :
Need to modify Manifest.MF file under CA module :
DafEar.Admin DCS-UI.Versioned BIZUI PubPortlet DCS-UI.SiteAdmin.Versioned SiteAdmin.Versioned DCS.Versioned DCS-UI DAF.Endeca.Index.Versioned DCS.Endeca.Index.Versioned DCS.Endeca.Index.SKUIndexing modules.PARALCommonModule modules.PARALModule modules.PARALMerchModule modules.Feed.Import modules.Feed.Export modules.Commerce
Primary Table :
select count(*) from paf_folder
select count(*) from paf_device_outputs
select count(*) from paf_display_modes
select count(*) from paf_community
select count(*) from paf_template
select count(*) from paf_page_template
select count(*) from paf_page
select count(*) from paf_layout
select count(*) from paf_style
select count(*) from paf_col_palette
select count(*) from paf_title_template
select count(*) from paf_region_def
select count(*) from paf_region
select count(*) from paf_gear_modes
select count(*) from paf_gear_def
select count(*) from paf_gear
select count(*) from paf_gear_param
select count(*) from paf_base_comm_role
select count(*) from paf_comm_template
select count(*) from paf_ct_folder
select count(*) from paf_ct_page
select count(*) from paf_ct_region
select count(*) from paf_ct_gear
select count(*) from paf_ct_alt_gear
InternalProfileRepository :
----------------------------
nohup startSQLRepository -m DPS.InternalUsers -s pub_export -encoding ISO-8859-1 -repository /atg/userprofiling/InternalProfileRepository -noTransaction -export all exported_internal_profile_dump.xml > exported_internal_profile_dump_log.txt &
Note :
Not exported Items:
-------------------
workbench
workbenchTile
workbenchReport
workbenchReportItem
Primary Table :
select count(*) from dpi_user
select count(*) from dpi_contact_info
select count(*) from dpi_mailing
select count(*) from dpi_mail_batch
select count(*) from dpi_mail_server
select count(*) from dpi_role
select count(*) from dpi_access_right
select count(*) from dpi_organization
select count(*) from dpi_folder
select count(*) from wb_workbench
select count(*) from wb_tile
select count(*) from wb_report
select count(*) from wb_report_item
select count(*) from wb_report_msg
select count(*) from dsi_profile_slot
select count(*) from dsi_ind_scenario
select count(*) from dsi_coll_scenario
select count(*) from dsi_scenario_info
select count(*) from dsi_scen_mig_info
select count(*) from dsi_template_info
select count(*) from dsi_coll_trans
select count(*) from dsi_ind_trans
select count(*) from dsi_deletion
select count(*) from dsi_migration
select count(*) from dsi_xref
ViewMappingRepository :
------------------------
nohup startSQLRepository -m modules.CA -s pub_export -encoding ISO-8859-1 -repository /atg/web/viewmapping/ViewMappingRepository -noTransaction -export all exported_viewMapping_dump.xml > exported_viewMapping_dump_log.txt &
Primary Table:
select count(*) from vmap_mode
select count(*) from vmap_fh
select count(*) from vmap_im
select count(*) from vmap_ivm
select count(*) from vmap_pvm
select count(*) from vmap_iv
select count(*) from vmap_pv
select count(*) from vmap_ivattrdef
select count(*) from vmap_pvattrdef
select count(*) from vmap_attrval
Import scripts/commands
The following repositories are to be imported which are export using above scripts in the XML file
-----------
1) /atg/multisite/SiteRepository (imported as part product catalog import )
2) /atg/seo/SEORepository
3) /atg/userprofiling/PersonalizationRepository
4) /atg/commerce/catalog/ProductCatalog
5) /atg/commerce/claimable/ClaimableRepository
6) /atg/commerce/locations/LocationRepository
7) /com/xx/giftregistry/repo/GiftRegistryRepository
8) /com/xx/newsLetter/NewsSubscriptionRepository
9) /com/xx/repo/BrandNumbersRepository
10) /com/xx/common/content/statictext/repo/ContentRepository
From PUB (6):
-------------
1) /atg/systemconfiguration/SystemConfigurationRepository
2) /atg/dynamo/security/AdminSqlRepository
3) /atg/portal/framework/PortalRepository
4) /atg/userprofiling/InternalProfileRepository
5) /atg/web/viewmapping/ViewMappingRepository
6) /atg/epub/file/PublishingFileRepository
Import Queries To PUB Schema :
----------------------------------
InternalProfileRepository :
---------------------------
nohup startSQLRepository -m DPS.InternalUsers -s pub_import -repository /atg/userprofiling/InternalProfileRepository -import /opt/ATG/ATG10.2/home/bin/Exported_Files_On_1_20_2017/CorrectFiles/PUB/exported_internal_profile_dump.xml > imported_internal_profile_log.txt &
ProductCatalog :
------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/catalog/ProductCatalog -file /opt/ATG/ATG10.2/home/bin/Exported_Files_On_1_20_2017/CorrectFiles/CAT-A/exported_productCatalog_dump.xml -workspace ProductCatalogImport -nocheckin > imported_productCatalog_log.txt &
[info@2020-12-03 04:28:27.608] created new workspace: workspace:75800
[info@2020-12-03 04:28:27.614] parsing input data
CONTAINER:atg.adapter.gsa.sqlimport.ImportException; SOURCE:com.ctc.wstx.exc.WstxUnexpectedCharException: Illegal character ((CTRL-CHAR, code 26))
at [row,col {unknown-source}]: [28036853,45]
at atg.adapter.gsa.sqlimport.ImportParserImpl.parseFirsPass(ImportParserImpl.java:130)
at atg.adapter.gsa.sqlimport.SQLImporter.doImport(SQLImporter.java:1992)
at atg.adapter.gsa.sqlimport.SQLImporter.execute(SQLImporter.java:1353)
at atg.adapter.gsa.sqlimport.SQLImporter.main(SQLImporter.java:1306)
Caused by: com.ctc.wstx.exc.WstxUnexpectedCharException: Illegal character ((CTRL-CHAR, code 26))
at [row,col {unknown-source}]: [28036853,45]
at com.ctc.wstx.sr.StreamScanner.throwInvalidSpace(StreamScanner.java:666)
at com.ctc.wstx.sr.StreamScanner.throwInvalidSpace(StreamScanner.java:651)
at com.ctc.wstx.sr.BasicStreamReader.readCDataPrimary(BasicStreamReader.java:4226)
at com.ctc.wstx.sr.BasicStreamReader.nextFromTreeCommentOrCData(BasicStreamReader.java:3285)
at com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2801)
at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1065)
at org.codehaus.stax2.ri.Stax2EventReaderImpl.nextEvent(Stax2EventReaderImpl.java:255)
at atg.adapter.gsa.sqlimport.ImportParserImpl.parseFirsPass(ImportParserImpl.java:112)
... 3 more
Solution :
sed -ie 's/few/asd/g' hello.txt
Run below on increasing maxConnection @JTData component at server layer and also
Need to add Modules.CA manifest file with below
ATG-Required: DafEar.Admin DCS-UI.Versioned BIZUI PubPortlet DCS-UI.SiteAdmin.Versioned SiteAdmin.Versioned DCS.Versioned DCS-UI DAF.Endeca.Index.Versioned DCS.Endeca.Index.Versioned DCS.Endeca.Index.SKUIndexing modules.PARALCommonModule modules.PARALModule modules.PARALMerchModule modules.Feed.Import modules.Feed.Export modules.Commerce
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/catalog/ProductCatalog -file /opt/ATG/ATG10.2/home/bin/exported_productCatalog_dump_22_1_2017_withRemovedChars.xml -workspace ProductCatalogImport -nocheckin -batchSize 5000 > /opt/ATG/ATG10.2/home/bin/imported_productCatalog_log_siva1.txt &
@163 box
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/catalog/ProductCatalog -file /opt/ATG/ATG10.2/home/bin/Apparao/exported_productCatalog_dump_22_1_2017_withRemovedChars.xml -workspace ProductCatalogImport1 -nocheckin -batchSize 5000 > /opt/ATG/ATG10.2/home/bin/imported_productCatalog_log_apparao_24_1.txt &
workspace :76800
LocationRepository :
---------------------
./startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/locations/LocationRepository -file exported_location_dump.xml -workspace location_import_workspace
**** Error Thu Dec 03 04:23:13 IST 2020 1606949593882 / Unable to set configured property "/com/xx/feed/export/listener/IncrementalPriceItemIndexingListener.incrementalQueueRepository" atg.nucleus.ConfigurationException: Unable to resolve component /atg/search/repository/IncrementalItemQueueRepository
**** Error Thu Dec 03 04:23:13 IST 2020 1606949593915 / Unable to set configured property "/atg/search/SynchronizationInvoker.port" atg.nucleus.ConfigurationException: Unable to resolve component /atg/search/Constants.null
**** Error Thu Dec 03 04:23:13 IST 2020 1606949593916 / Unable to set configured property "/atg/search/SynchronizationInvoker.host" atg.nucleus.ConfigurationException: Unable to resolve component /atg/search/Constants.null
**** info Thu Dec 03 04:23:13 IST 2020 1606949593967 /atg/epub/deployment/DeploymentManager Resolving reference to /atg/deployment/DeploymentManager
[info@2020-12-03 04:23:14.089] created new workspace: workspace:75701
[info@2020-12-03 04:23:14.096] parsing input data
[info@2020-12-03 04:23:14.569] importing 184 items.
[info@2020-12-03 04:23:14.569] checking out versioned items in batch
[info@2020-12-03 04:23:15.599] performing add/update/removes
[info@2020-12-03 04:23:16.825] Phase 2 of 2: 100%
[info@2020-12-03 04:23:16.826] checkIn: workspace:75701
java.lang.AbstractMethodError: atg.adapter.version.VersionRepository.getVersionItemsInLine(Latg/versionmanager/DevelopmentLine;ZZZ)Ljava/util/Iterator;
at atg.adapter.secure.GenericSecuredMutableVersionRepository.getVersionItemsInLine(GenericSecuredMutableVersionRepository.java:883)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.get(DevelopmentLineRepositoryImpl.java:547)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.getAllAssetVersions(DevelopmentLineRepositoryImpl.java:376)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.getAllAssetVersions(DevelopmentLineRepositoryImpl.java:393)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.getAllAssetVersions(DevelopmentLineRepositoryImpl.java:351)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.getAllAssetVersionsCount(DevelopmentLineRepositoryImpl.java:404)
at atg.versionmanager.impl.WorkspaceRepositoryImpl.checkInAll(WorkspaceRepositoryImpl.java:1490)
at atg.adapter.gsa.sqlimport.SQLImporter.checkIn(SQLImporter.java:2512)
at atg.adapter.gsa.sqlimport.SQLImporter.doImport(SQLImporter.java:2043)
at atg.adapter.gsa.sqlimport.SQLImporter.execute(SQLImporter.java:1353)
at atg.adapter.gsa.sqlimport.SQLImporter.main(SQLImporter.java:1306)
Solution: Again done @ 163 box. So issue.
only media-extranal item port :
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/catalog/ProductCatalog -file /opt/ATG/ATG10.2/home/bin/exported_media_external_dump.xml -workspace Only_MediaItem_Import -nocheckin > Only_MediaItem_Import.txt &
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/catalog/ProductCatalog -file /opt/ATG/ATG10.2/home/bin/exported_media1_external_dump.xml -workspace exported_media1_external_dump -nocheckin > exported_media1_external_dump.txt &
SEORepository :
---------------
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/seo/SEORepository -file /opt/ATG/ATG10.2/home/bin/Exported_Files_On_1_20_2017/CorrectFiles/CAT-A/exported_SEO_dump.xml -workspace SEO_Import > imported_SEO_log.txt &
log:
[info@2020-12-05 04:46:48.034] parsing input data
[info@2020-12-05 04:46:48.323] importing 239 items.
[info@2020-12-05 04:46:48.323] checking out versioned items in batch
[info@2020-12-05 04:46:49.338] performing add/update/removes
[info@2020-12-05 04:46:50.472] Phase 2 of 2: 100%
[info@2020-12-05 04:46:50.472] checkIn: workspace:76100
java.lang.AbstractMethodError: atg.adapter.version.VersionRepository.getVersionItemsInLine(Latg/versionmanager/DevelopmentLine;ZZZ)Ljava/util/Iterator;
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.get(DevelopmentLineRepositoryImpl.java:547)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.getAllAssetVersions(DevelopmentLineRepositoryImpl.java:376)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.getAllAssetVersions(DevelopmentLineRepositoryImpl.java:393)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.getAllAssetVersions(DevelopmentLineRepositoryImpl.java:351)
at atg.versionmanager.impl.DevelopmentLineRepositoryImpl.getAllAssetVersionsCount(DevelopmentLineRepositoryImpl.java:404)
at atg.versionmanager.impl.WorkspaceRepositoryImpl.checkInAll(WorkspaceRepositoryImpl.java:1490)
at atg.adapter.gsa.sqlimport.SQLImporter.checkIn(SQLImporter.java:2512)
at atg.adapter.gsa.sqlimport.SQLImporter.doImport(SQLImporter.java:2043)
at atg.adapter.gsa.sqlimport.SQLImporter.execute(SQLImporter.java:1353)
at atg.adapter.gsa.sqlimport.SQLImporter.main(SQLImporter.java:1306)
Solution: Again run at 163 box. Assets are checked in.
SystemConfigurationRepository :
-------------------------------
nohup startSQLRepository -m modules.CA -s pub_import -repository /atg/systemconfiguration/SystemConfigurationRepository -import /opt/ATG/ATG10.2/home/bin/Exported_Files_On_1_20_2017/CorrectFiles/PUB/exported_sys_conf_dump.xml > imported_system_conf_log.txt &
AdminSqlRepository :
--------------------
nohup startSQLRepository -m modules.CA -s pub_import -repository /atg/dynamo/security/AdminSqlRepository -import /opt/ATG/ATG10.2/home/bin/Exported_Files_On_1_20_2017/CorrectFiles/PUB/exported_admin_sql_dump.xml > imported_AdminSql_log.txt &
PortalRepository :
------------------
nohup startSQLRepository -m modules.CA -s pub_import -repository /atg/portal/framework/PortalRepository -import /opt/ATG/ATG10.2/home/bin/Exported_Files_On_1_20_2017/CorrectFiles/PUB/exported_portal_dump.xml > imported_portal_log.txt &
PersonalizationRepository :version Repository, executed first as no checkin
---------------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/userprofiling/PersonalizationRepository -file /opt/ATG/ATG10.2/home/bin/Exported_Files_On_1_20_2017/CorrectFiles/CAT-A/exported_personalization_dump.xml -workspace Personalization_Import > imported_personalization_log.txt &
ViewMappingRepository :
-----------------------
nohup startSQLRepository -m modules.CA -s pub_import -repository /atg/web/viewmapping/ViewMappingRepository -import /opt/ATG/ATG10.2/home/bin/Exported_Files_On_1_20_2017/CorrectFiles/PUB/exported_viewMapping_dump.xml > imported_viewMapping_log.txt &
ClaimableRepository :
---------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/claimable/ClaimableRepository -file /opt/ATG/ATG10.2/home/bin/Apparao/exported_claimable_dump.xml -workspace Claimable_import > imported_Claimable_log_apparao.txt &
LocationRepository : agin run at 163 box
---------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /atg/commerce/locations/LocationRepository -file /opt/ATG/ATG10.2/home/bin/Apparao/exported_location_dump.xml -workspace location_import_workspace > imported_location_log_apparao.txt &
GiftRegistryRepository :
------------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /com/xx/giftregistry/repo/GiftRegistryRepository -file /opt/ATG/ATG10.2/home/bin/Apparao/exported_giftregistry_dump.xml -workspace GiftRegistry_import > imported_giftRegistry_log_apparao.txt &
NewsSubscriptionRepository :
-----------------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /com/xx/newsLetter/NewsSubscriptionRepository -file /opt/ATG/ATG10.2/home/bin/Apparao/exported_newsSubscription_dump.xml -workspace NewsSubscription_import > imported_newsSubscription_log_apparao.txt &
BrandNumbersRepository :
------------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /com/xx/repo/BrandNumbersRepository -file /opt/ATG/ATG10.2/home/bin/Apparao/exported_brandnumbers_dump.xml -workspace BrandNumbers_imported > imported_brandnumber_log_apparao.txt &
ContentRepository :
-------------------
nohup startSQLImport -m modules.CA -s pub_import -repository /com/xx/common/content/statictext/repo/ContentRepository -file /opt/ATG/ATG10.2/home/bin/Apparao/exported_content_dump.xml -workspace Content_imported > imported_content_log_apparao.txt &
No comments:
Post a Comment