Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 991 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.2765. Min Act: -0.1700

Data Index: 1023869 (Open Web Text)

Max Activating Token Index: 84

Text #1

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.2403. Min Act: -0.1700

Data Index: 2294097 (Open Web Text)

Max Activating Token Index: 76

Text #2

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.2295. Min Act: -0.1700

Data Index: 2898151 (Open Web Text)

Max Activating Token Index: 77

Text #3

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.2148. Min Act: -0.1700

Data Index: 2879227 (Open Web Text)

Max Activating Token Index: 86

Text #4

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.2154. Min Act: -0.1700

Data Index: 5453467 (Open Web Text)

Max Activating Token Index: 77

Text #5

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.2050. Min Act: -0.1700

Data Index: 5764764 (Open Web Text)

Max Activating Token Index: 72

Text #6

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.1823. Min Act: -0.1700

Data Index: 4246084 (Open Web Text)

Max Activating Token Index: 77

Text #7

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.1842. Min Act: -0.1700

Data Index: 2294103 (Open Web Text)

Max Activating Token Index: 78

Text #8

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.1732. Min Act: -0.1700

Data Index: 2123188 (Open Web Text)

Max Activating Token Index: 77

Text #9

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.1792. Min Act: -0.1700

Data Index: 4147148 (Open Web Text)

Max Activating Token Index: 90

Text #10

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.1695. Min Act: -0.1700

Data Index: 5460422 (Open Web Text)

Max Activating Token Index: 97

Text #11

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.1715. Min Act: -0.1700

Data Index: 2330811 (Open Web Text)

Max Activating Token Index: 79

Text #12

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.1669. Min Act: -0.1700

Data Index: 502156 (Open Web Text)

Max Activating Token Index: 80

Text #13

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.1589. Min Act: -0.1700

Data Index: 937485 (Open Web Text)

Max Activating Token Index: 78

Text #14

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.1621. Min Act: -0.1700

Data Index: 8045724 (Open Web Text)

Max Activating Token Index: 81

Text #15

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.1529. Min Act: -0.1700

Data Index: 3634095 (Open Web Text)

Max Activating Token Index: 76

Text #16

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.1483. Min Act: -0.1700

Data Index: 5213703 (Open Web Text)

Max Activating Token Index: 72

Text #17

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.1518. Min Act: -0.1700

Data Index: 5595831 (Open Web Text)

Max Activating Token Index: 65

Text #18

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.1460. Min Act: -0.1700

Data Index: 608322 (Open Web Text)

Max Activating Token Index: 91

Text #19

Max Range: 1.2765. Min Range: -1.2765

Max Act: 1.1496. Min Act: -0.1700

Data Index: 6947648 (Open Web Text)

Max Activating Token Index: 85