Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2164 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.2782. Min Act: -0.1700

Data Index: 4423604 (Open Web Text)

Max Activating Token Index: 735

Text #1

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.2481. Min Act: -0.1700

Data Index: 3755757 (Open Web Text)

Max Activating Token Index: 462

Text #2

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1988. Min Act: -0.1700

Data Index: 856390 (Open Web Text)

Max Activating Token Index: 43

Text #3

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1770. Min Act: -0.1700

Data Index: 5706168 (Open Web Text)

Max Activating Token Index: 134

Text #4

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1841. Min Act: -0.1700

Data Index: 7887432 (Open Web Text)

Max Activating Token Index: 9

Text #5

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1925. Min Act: -0.1700

Data Index: 3650066 (Open Web Text)

Max Activating Token Index: 18

Text #6

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1930. Min Act: -0.1700

Data Index: 8616959 (Open Web Text)

Max Activating Token Index: 15

Text #7

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1834. Min Act: -0.1700

Data Index: 8409067 (Open Web Text)

Max Activating Token Index: 16

Text #8

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1947. Min Act: -0.1700

Data Index: 5034317 (Open Web Text)

Max Activating Token Index: 663

Text #9

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1556. Min Act: -0.1700

Data Index: 5019147 (Open Web Text)

Max Activating Token Index: 176

Text #10

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1732. Min Act: -0.1700

Data Index: 2745039 (Open Web Text)

Max Activating Token Index: 445

Text #11

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1832. Min Act: -0.1700

Data Index: 243910 (Open Web Text)

Max Activating Token Index: 618

Text #12

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1716. Min Act: -0.1700

Data Index: 7290973 (Open Web Text)

Max Activating Token Index: 351

Text #13

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1724. Min Act: -0.1700

Data Index: 8233716 (Open Web Text)

Max Activating Token Index: 396

Text #14

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1683. Min Act: -0.1700

Data Index: 7351477 (Open Web Text)

Max Activating Token Index: 7

Text #15

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1569. Min Act: -0.1700

Data Index: 3552555 (Open Web Text)

Max Activating Token Index: 550

Text #16

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1562. Min Act: -0.1700

Data Index: 3572906 (Open Web Text)

Max Activating Token Index: 489

Text #17

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1450. Min Act: -0.1700

Data Index: 5208947 (Open Web Text)

Max Activating Token Index: 365

Text #18

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1448. Min Act: -0.1700

Data Index: 3038682 (Open Web Text)

Max Activating Token Index: 20

Text #19

Max Range: 2.2782. Min Range: -2.2782

Max Act: 2.1488. Min Act: -0.1700

Data Index: 1175183 (Open Web Text)

Max Activating Token Index: 38