# ShareAndPull SHAP [![Scrutinizer Code Quality](https://scrutinizer-ci.com/g/LancasterSolutions/ShareAndPull/badges/quality-score.png?b=master&s=24f58a3fa18e1d7e14c3b1a2a33ba43e75a10cf2)](https://scrutinizer-ci.com/g/LancasterSolutions/ShareAndPull/?branch=master) > Documentation & Useful notes Share and Pull is a tool that will let you synchronize your database with others. It's meant to be a incremental synchronization system. SHAP is divided in lists. Each list has several data types ("email","name","..."). A list has always a primary key. The primary key is unique so when we receive a message with the same primary key we will overwrite the original message with the new content. SHAP lets you push up to 3 messages per request. When you do a pull we will send you a number of messages around 300. If there is only 5 new messages from the time that you do the request only 5 messages will be send. This way you know that if you ask for the last hour and no new messages come, you are up to date. If 300 or more messages are present in the request that means you can still pull more using the latest "timestamp" from the last message. Once the messages have been pushed to our system we queue them for validation. This process consumes resources and is not instant like pull request. It might take some minutes before we add the messages to the list. SHAP is meant to be used based on "timestamps". This means that every time a message is created or a message is updated we set a timestamp to that message. This way we know that it's a new information. When you pull, use always the filter "timestamp" so you can follow the new messages with incremental pulls based on it. For example you can start the synchronization using timestamp 1. Then pull again with the timestamp of the last element of the request. This way you will be getting messages in order. When you get 0 messages that will mean that you have ended the synchronization. Then you will be up to date with the list. > Authentication The authentication in shap is based on API keys. You will find your API key in the list management section. Once you have your key you will have to send it on every call to our API. With this API we can identify both the list and the user that is using it. Example URL GET /api/data/pull/1446503522 Mandatory Headers ```html [ "partner-api-key" => "you api key for the desired list" , "Content-Type" => "application/json", "Accept" => "application/json", ] ``` ## Detail The DETAIL method will give you the structural information of a list. You will know the amount of columns and details of each. It will know the list based on the API key. URL GET /api/data/detail Mandatory Headers ```html [ "partner-api-key" => "you api key for the desired list" , "Content-Type" => "application/json", "Accept" => "application/json", ] ``` In case something went wrong you will find a status code 4xx or 5xx. Status code 200 is sent when correct response is given. A example of 200 "OK" response ```html { "listName": "BlackListedEmails", "listId": 1, "listDetails": [ "email" ] } ``` ## Pull The PULL method will return back messages from the desired list. The list will be chosen upon API key send on the request. The messages will come in order by timestamp. The calls to the PULL method consume Credits. Each message returned by the call will remove from your balance 1 credit/s. The PULL method will give you all the messages more recent than the specified timestamp. The limit of messages for a pull request on the server is set to 300. In case that there are more messages with the last timestamp the will also come so that you can ensure that you have all the messages up to the last timestamp of the last message. PARAMETERS Timestamp: must be a positive integer. It should not exceed the min or max in the unix_timestamp function URL GET /api/data/pull/1446503522 Mandatory Headers ```json [ "partner-api-key" => "you api key for the desired list" , "Content-Type" => "application/json", "Accept" => "application/json", ] ``` In case something went wrong you will find a status code 4xx or 5xx. Status code 200 is sent when correct response is given. A example of 200 "OK" response ```json { "1": { "email": "mateo@blacklisted.com", "timestamp": 1473245469 }, "2": { "email": "mateo@fakeemail.com", "timestamp": 1473245469 } } ``` ## Push The PUSH method will add to the validation queue of the list the messages you push. This process will validate the data. Validations can be very complex and some data types can gave multiple regular expressions to match. This process might take a couple of minutes. The calls to the PUSH method grant Credits. Each message validated by the queue will add to your balance 1 credit/s. The PUSH method will support push of messages to a list up to a max number of 3 messages. In case that more messages are sent the API will return a 400 "bad request". URL GET /api/data/push Mandatory Headers ``` [ "partner-api-key" => "you api key for the desired list" , "Content-Type" => "application/json", "Content-Length" => strlen($body), "Accept" => "application/json", ] ``` In case something went wrong you will find a status code 4xx or 5xx. Status code 200 is sent when correct response is given. A example of 200 "OK" response. The response confirms the amount of messages queued ``` { "queued": 2 } ``` ## Worker - Execute: ``` php bin/console DataBundle:QueueManager --env=prod --no-debug ``` - It's a keep alive worker, you can check the PID in linux with a cron - Ideally build a daemon - Extracts info from AWS and process it after validation - Uses validation interface \DataBundle\Service\Validation