Welcome to AIFS Documentation

This project is hosted by Digital Oversight.

The code is open source, and available on Github.

About AIFS

This framework implement large scale information gathering and analysis in order to provide valuable intelligence to a third party software. Using multiple sources, this software calculate the risk factors related to a decision, regardeless of the initial objective and the resulting actions.

AIFS tries to implement general patterns related to machine learning and intelligence classification. The core functionalities include libraries related to public content information, digital network information, geographical and human information.

License

Copyright (c) 2016-2017, Digital Oversight AIFS LICENSE

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE X CONSORTIUM BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Except as contained in this notice, the name of the Digital Oversight shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Software without prior written authorization from Digital Oversight.

Commercial use, distribution, merge, sublicense and/or sale of the Software must display a valid AIFS coin hash along with this license.

Downloads

Version Description
master.zip aifs core current version.
1.03 - 30/05/2017  
aifs-1.03.zip aifs core 1.03
aifs-extends-1.03.zip aifs extension 1.03
1.02 - 12/01/2016  
aifs-1.02.zip aifs core 1.02

How to install

# git clone https://github.com/digitaloversight/aifs.git

# mysql -u root -p < ./aifs/schema/aifs-base-1.02.sql
mysql> insert into user set Host='localhost', User='aifs', Password=PASSWORD('');

mysql> flush privileges;

mysql> grant all on aifs.*  to 'aifs'@'localhost';

Configuration

Make sure you reviwed the configuration on the following files

``/config/config.php``

``/config/tool/DomainSelector.php``

Extends

AIFS comes with extensions, please take a look at the aifs-extends repository available on Github.

Configuration

If you do not load the web administration panel, you will need two things to enable basic features of AIFS : a keyword dictionary and an url list.

Keyword dictionary

If you wish to enable semantic indexing, you need to load a dictionary. We provide the English-French words list in the aifs-extends repository. Load it on a freshly installed aifs-base-1.02.sql schema using this command :

# git clone https://github.com/digitaloversight/aifs-extends.git

# tar -xvzf ./aifs-extends/data/osint_keyword_EN_FR_102.sql.tar.gz

# mysql -u root -p aifs < ./aifs-extends/data/osint_keyword_EN_FR_102.sql

URL list

If you do not use the web administration panel to provide urls, you will need to fill the osint_url manually.

mysql> insert into osint_url set url='https://twitter.com/infainit';

Routines

OSINT routines

Fetch Version routine/osint_fetch_version.php

Perform a version check with a randomly choosen url in the user filled url table.

Changes routine/osint_changes.php

Execute a comparison algorithm between two saved version of a document stored.

Title routine/osint_title.php

Extract title from html content.

Tag count routine/osint_tagcount.php

Precalculate the number of urls linked for all tags stored in the osint_tags

Table Rotation routine/osint_table_rotation.php

Rotate the web content present into the main version content by renaming the sql table.

DNINT routines

Semantic Analysis routine/dnint_url_semantic.php

Apply semantic analysis against html parsed content.

Oubound* routine/dnint_oubound.php

Get the outbound links on an html version.

NS Check routine/dnint_ns_check.php

Perform an NS check on a saved url.

Google PR routine/dnint_google_pr.php

Fetch the google pr for a specified url.

FININT routines

Currency routine/finint_currency.php

Responses

Normal Responses

Routines display different json of responses depending on the action performed.

Error code Message Notes
200    
200001 No valid record to parse.  
200002 Current record already processed.  
200003 No action needed.  

Error Responses

AIFS provide json responses along with a 500 HTTP response code when an error is triggered by the routines scripts.

Error code Message Notes
500    
500001 Cannot open sql connection on host. Your database parameters are wrong or your database is down.
500002 Impossible to select database on host.  
500003 Cannot execute query without connection. Lost connection handle.
500004 We are unable to execute your request. Query syntax error. This error trigger higher log level.
500005 Query not executed. Data extraction errors, data type errors.

AIFS ocr

This software includes a basic OCR (optical caracter reader) binary designed to parse visible text from image files. As of version 1.02, the ocr feature is not linked with the semantic indexer nor with any image feeder available in the aifs-extends package.

Requirements

OpenCV-2.4.0

tesseract-3.01

leptonica-1.69

Builds are made are available for debian linux machines.