Welcome to AIFS Documentation¶
This project is hosted by Digital Oversight.
The code is open source, and available on Github.
This framework implement large scale information gathering and analysis in order to provide valuable intelligence to a third party software. Using multiple sources, this software calculate the risk factors related to a decision, regardeless of the initial objective and the resulting actions.
AIFS tries to implement general patterns related to machine learning and intelligence classification. The core functionalities include libraries related to public content information, digital network information, geographical and human information.
Copyright (c) 2016-2017, Digital Oversight AIFS LICENSE
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE X CONSORTIUM BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Except as contained in this notice, the name of the Digital Oversight shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Software without prior written authorization from Digital Oversight.
Commercial use, distribution, merge, sublicense and/or sale of the Software must display a valid AIFS coin hash along with this license.
|master.zip||aifs core current version.|
|1.03 - 30/05/2017|
|aifs-1.03.zip||aifs core 1.03|
|aifs-extends-1.03.zip||aifs extension 1.03|
|1.02 - 12/01/2016|
|aifs-1.02.zip||aifs core 1.02|
How to install¶
# git clone https://github.com/digitaloversight/aifs.git # mysql -u root -p < ./aifs/schema/aifs-base-1.02.sql
mysql> insert into user set Host='localhost', User='aifs', Password=PASSWORD(''); mysql> flush privileges; mysql> grant all on aifs.* to 'aifs'@'localhost';
Make sure you reviwed the configuration on the following files
If you do not load the web administration panel, you will need two things to enable basic features of AIFS : a keyword dictionary and an url list.
If you wish to enable semantic indexing, you need to load a dictionary. We provide the English-French words list in the aifs-extends repository.
Load it on a freshly installed
aifs-base-1.02.sql schema using this command :
# git clone https://github.com/digitaloversight/aifs-extends.git # tar -xvzf ./aifs-extends/data/osint_keyword_EN_FR_102.sql.tar.gz # mysql -u root -p aifs < ./aifs-extends/data/osint_keyword_EN_FR_102.sql
If you do not use the web administration panel to provide urls, you will need to fill the osint_url manually.
mysql> insert into osint_url set url='https://twitter.com/infainit';
Perform a version check with a randomly choosen url in the user filled url table.
Execute a comparison algorithm between two saved version of a document stored.
Extract title from html content.
Precalculate the number of urls linked for all tags stored in the osint_tags
Rotate the web content present into the main version content by renaming the sql table.
Apply semantic analysis against html parsed content.
Get the outbound links on an html version.
Perform an NS check on a saved url.
Fetch the google pr for a specified url.
Routines display different json of responses depending on the action performed.
|200001||No valid record to parse.|
|200002||Current record already processed.|
|200003||No action needed.|
AIFS provide json responses along with a 500 HTTP response code when an error is triggered by the routines scripts.
|500001||Cannot open sql connection on host.||Your database parameters are wrong or your database is down.|
|500002||Impossible to select database on host.|
|500003||Cannot execute query without connection.||Lost connection handle.|
|500004||We are unable to execute your request.||Query syntax error. This error trigger higher log level.|
|500005||Query not executed.||Data extraction errors, data type errors.|
This software includes a basic OCR (optical caracter reader) binary designed to parse visible text from image files. As of version 1.02, the ocr feature is not linked with the semantic indexer nor with any image feeder available in the aifs-extends package.
Builds are made are available for debian linux machines.