Implemented 'fields_from' for dynamic loading of form fields.

doc/MANUAL.md

@@ -39,7 +39,11 @@ This is the manual for version %%VERSION%%.
     - [Text](#field_types_text)
     - [Password](#field_types_password)
     - [File](#field_types_file)
-1. [Dynamic forms](#dynamic_forms)
+1. [Dynamic forms](#dynform)
+    - [Supported dynamic form parts](#dynform_support)
+    - [Dynamic select and radio options](#dynform_selectradio)
+    - [Dynamic fields](#dynform_fields)
+    - [Notes](#dynform_notes)
 1. [Output](#output)
     - [Output types](#output_types)
     - [Exit codes](#output_exitcodes)
@@ -649,7 +653,7 @@ Structurally, they are made up of the following elements:
       **Default:** `nobody`.
 
     - **`fields`**: List of fields in the form. Each field is a dictionary.
-      **Required**, **List of dictionaries**.
+      **Optional**, **List of dictionaries**.
 
         - **`name`**: The name of the field. This is what is passed as an
           environment variable to the callback. **Required**, **String**.
@@ -677,6 +681,11 @@ Structurally, they are made up of the following elements:
         - **`...`**: Other options, which depend on the type of field.  For
           more information, see [Field types](#field_types). **Optional**.
 
+    - **`fields_from`**: Path to a file or executable from which to read the
+      list of fields. See the **fields** option for information on fields. See
+      the [Dynamic forms](#dynamic_forms) chapter for more information.
+      **Optional**, **String**.
+
 - **`users`**: A dictionary of users where the key is the username and the
   value is the plain text password. This field is not required. **Dictionary**.
 
@@ -1043,7 +1052,7 @@ For example:
 
 
 
-## <a name="dynamic_forms">Dynamic Forms</a>
+## <a name="dynform">Dynamic Forms</a>
 
 Often it won't be enough to have staticly defined fields in forms. For
 example, you may  want the user to be able to select from a dynamically
@@ -1053,9 +1062,20 @@ Scriptform offers a way to fill in parts of a form definition file dynamically
 from extern files or scripts. This is done at runtime when showing or
 validating the form, so no restarting of scriptform is required.
 
-When it's possible to dynamically load parts of a form, the form definition
+### <a name="dynform_support">Supported dynamic form parts</a>
-haev a `_from` option. For example, normally you'd use the `options` element
+
-for a `select` field in  a form, like so:
+Currently the following parts of form definitions support dynamic form parts:
+
+* `forms.X.fields_from`: Form's fields are read from a file or script.
+*  Fields of type `select`. Options for dropdowns are read from a file or
+   script.
+*  Fields of type `radio`. Options for radio buttons are read from a file or
+   script.
+
+### <a name="dynform_selectradio">Dynamic select and radio options</a>
+
+Normally you'd use the `options` element for a `select` field in  a form, like
+so:
 
     ...
     "fields": [
@@ -1102,19 +1122,71 @@ The script `form_loadcsv_target_db.sh` could look like:
     ]
     END_TEXT
 
-Notes:
+Dynamic options for radio buttons works exactly the same, except the type
+will be `radio`:
+
+    "type": "select",
+    "options": [
+      ["empty", "Empty database"],
+      ["dev", "Development test database"],
+      ["ua", "Acceptance database"]
+    ]
+
+### <a name="dynform_fields">Dynamic fields</a>
+
+You can also generate all the fields in a form dynamically. For example, the
+following form definition will read all the fields from `form_dyn_fields.sh`:
+
+    {
+        "name": "dyn_fields",
+        "title": "Dynamic fileds",
+        "description": "All the fields in this form are dynamically read from a script.",
+        "submit_title": "Import",
+        "script": "job_import.sh",
+        "fields_from": "form_dyn_fields.sh"
+    }
+
+The script `form_dyn_fields.sh` could look something like:
+
+    #!/bin/sh
+
+    OPTIONS=$(cat <<'END_HEREDOC'
+    [
+        ["test", "Test DB"],
+        ["acc", "Acc DB"],
+        ["prod", "Prod DB"]
+    ]
+    END_HEREDOC
+    )
+
+    cat << END_TEXT
+    [
+        {
+            "name": "target_db",
+            "title": "Database to import to",
+            "type": "radio",
+            "options": $OPTIONS
+        },
+        {
+            "name": "sql_file",
+            "title": "SQL file",
+            "type": "file"
+        }
+    ]
+    END_TEXT
+
+### <a name="dynform_notes">Notes</a>
 
 * The executable bit must be set in order for scriptform to execute the file.
   Otherwise, the contents of the file is read.
 * Executable scripts are *always* executed as the user scriptform runs at!
   While its not possible for the user to inject anything into the script, you
   should still be careful with what the scripts do.
-
+* Dynamic parts of a form are loaded / executed each time the form is
-### Supported dynamic form parts
+  referenced, so may be read or executed often. E.g. when loading the form,
-
+  when listing the forms, when showing the form, when submitting the form,
-Currently the following parts of form definitions support dynamic form parts:
+  when validatinng the form, etc. You should make sure executables run
-
+  quickly.
-* `fields`: `type` = `select`.
 
 
 ## <a name="output">Output</a>

examples/dynamic_forms/dynamic_forms.json

@@ -2,17 +2,17 @@
     "title": "Dynamic forms",
     "forms": [
         {
-            "name": "import",
+            "name": "dyn_options",
-            "title": "Import CSV data",
+            "title": "Dynamic options",
-            "description": "Import CSV into a database",
+            "description": "The values for the target database are dynamically read from a script.",
             "submit_title": "Import",
             "script": "job_import.sh",
             "fields": [
                 {
                     "name": "target_db",
-                    "title": "Database to import to",
+                    "title": "Target database to import to",
                     "type": "radio",
-                    "options_from": "form_import_target_dbs.sh"
+                    "options_from": "form_dyn_options_target_db.sh"
                 },
                 {
                     "name": "sql_file",
@@ -20,6 +20,14 @@
                     "type": "file"
                 }
             ]
+        },
+        {
+            "name": "dyn_fields",
+            "title": "Dynamic fileds",
+            "description": "All the fields in this form are dynamically read from a script.",
+            "submit_title": "Import",
+            "script": "job_import.sh",
+            "fields_from": "form_dyn_fields.sh"
         }
     ]
 }

examples/dynamic_forms/form_dyn_fields.sh

@@ -0,0 +1,26 @@
+#!/bin/sh
+
+OPTIONS=$(cat <<'END_HEREDOC'
+[
+    ["test", "Test DB"],
+    ["acc", "Acc DB"],
+    ["prod", "Prod DB"]
+]
+END_HEREDOC
+)
+
+cat << END_TEXT
+[
+    {
+        "name": "target_db",
+        "title": "Database to import to",
+        "type": "radio",
+        "options": $OPTIONS
+    },
+    {
+        "name": "sql_file",
+        "title": "SQL file",
+        "type": "file"
+    }
+]
+END_TEXT

examples/dynamic_forms/job_import.sh

@@ -1,6 +1,5 @@
 #!/bin/sh
 
-
 MYSQL_DEFAULTS_FILE="my.cnf"
 MYSQL="mysql --defaults-file=$MYSQL_DEFAULTS_FILE"
 

src/formdefinition.py

@@ -20,13 +20,15 @@ class FormDefinition(object):
     for validation of the form values.
     """
     def __init__(self, name, title, description, fields, script,
-                 default_value=None, output='escaped', hidden=False,
+                 fields_from=None, default_value=None, output='escaped',
-                 submit_title="Submit", allowed_users=None, run_as=None):
+                 hidden=False, submit_title="Submit", allowed_users=None,
+                 run_as=None):
         self.name = name
         self.title = title
         self.description = description
         self.fields = fields
         self.script = script
+        self.fields_from = fields_from
         self.default_value = default_value
         self.output = output
         self.hidden = hidden
@@ -34,7 +36,20 @@ class FormDefinition(object):
         self.allowed_users = allowed_users
         self.run_as = run_as
 
-        self.validate_field_defs(self.fields)
+        self.validate_field_defs(self.get_fields())
+
+    def get_fields(self):
+        """
+        Return the fields for the form either from statically defined fields in
+        the form definition, or dynamically from an externally executable
+        script.
+        """
+        if self.fields is not None:
+            return self.fields
+        elif self.fields_from is not None:
+            return runscript.from_file(self.fields_from)
+        else:
+            raise ValueError("Missing either 'fields' or 'fields_from' in '{}' form".format(self.name))
 
     def validate_field_defs(self, fields):
         """
@@ -52,7 +67,7 @@ class FormDefinition(object):
         """
         Return the field definition for `field_name`.
         """
-        for field in self.fields:
+        for field in self.get_fields():
             if field['name'] == field_name:
                 return field
         raise KeyError("Unknown field: {0}".format(field_name))
@@ -67,7 +82,7 @@ class FormDefinition(object):
         values = form_values.copy()
 
         # First make sure all required fields are there
-        for field in self.fields:
+        for field in self.get_fields():
             field_required = ('required' in field and
                               field['required'] is True)
             field_missing = (field['name'] not in form_values or
@@ -78,7 +93,7 @@ class FormDefinition(object):
                 )
 
         # Validate the field values, possible casting them to the correct type.
-        for field in self.fields:
+        for field in self.get_fields():
             field_name = field['name']
             if field_name in errors:
                 # Skip fields that are required but missing, since they can't

src/scriptform.py

@@ -83,8 +83,9 @@ class ScriptForm(object):
                 FormDefinition(form_name,
                                form['title'],
                                form['description'],
-                               form['fields'],
+                               form.get('fields', None),
                                script,
+                               fields_from=form.get("fields_from", None),
                                default_value=form.get('default_value', ""),
                                output=form.get('output', 'escaped'),
                                hidden=form.get('hidden', False),

src/webapp.py

@@ -165,7 +165,7 @@ def censor_form_values(form_def, form_values):
     Remove sensitive field values from form_values dict.
     """
     censored_form_values = copy.copy(form_values)
-    for field in form_def.fields:
+    for field in form_def.get_fields():
         if field['type'] == 'password':
             censored_form_values[field['name']] = '********'
     return censored_form_values
@@ -374,7 +374,7 @@ class ScriptFormWebApp(RequestHandler):
             name=form_def.name,
             fields=u''.join(
                 [render_field(f, errors.get(f['name'], []))
-                 for f in form_def.fields]
+                 for f in form_def.get_fields()]
             ),
             submit_title=form_def.submit_title
         )