Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1760 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.4587. Min Act: -0.1700

Data Index: 1949733 (Open Web Text)

Max Activating Token Index: 76

Text #1

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.4235. Min Act: -0.1700

Data Index: 8557846 (Open Web Text)

Max Activating Token Index: 590

Text #2

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.4030. Min Act: -0.1700

Data Index: 1180311 (Open Web Text)

Max Activating Token Index: 451

Text #3

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.4264. Min Act: -0.1700

Data Index: 8134340 (Open Web Text)

Max Activating Token Index: 132

Text #4

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.3880. Min Act: -0.1700

Data Index: 7647096 (Open Web Text)

Max Activating Token Index: 87

Text #5

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.4022. Min Act: -0.1700

Data Index: 1574219 (Open Web Text)

Max Activating Token Index: 518

Text #6

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.3995. Min Act: -0.1700

Data Index: 5850634 (Open Web Text)

Max Activating Token Index: 204

Text #7

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.4050. Min Act: -0.1700

Data Index: 4582178 (Open Web Text)

Max Activating Token Index: 556

Text #8

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.4004. Min Act: -0.1700

Data Index: 2497603 (Open Web Text)

Max Activating Token Index: 231

Text #9

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.3733. Min Act: -0.1700

Data Index: 2485773 (Open Web Text)

Max Activating Token Index: 174

Text #10

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.3746. Min Act: -0.1700

Data Index: 548243 (Open Web Text)

Max Activating Token Index: 986

Text #11

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.3576. Min Act: -0.1700

Data Index: 173282 (Open Web Text)

Max Activating Token Index: 210

Text #12

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.3060. Min Act: -0.1700

Data Index: 5334177 (Open Web Text)

Max Activating Token Index: 314

Text #13

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.3017. Min Act: -0.1700

Data Index: 2476539 (Open Web Text)

Max Activating Token Index: 619

Text #14

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.2850. Min Act: -0.1700

Data Index: 6250414 (Open Web Text)

Max Activating Token Index: 759

Text #15

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.2902. Min Act: -0.1700

Data Index: 6216447 (Open Web Text)

Max Activating Token Index: 780

Text #16

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.2894. Min Act: -0.1700

Data Index: 6365930 (Open Web Text)

Max Activating Token Index: 544

Text #17

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.2857. Min Act: -0.1700

Data Index: 1354919 (Open Web Text)

Max Activating Token Index: 97

Text #18

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.2870. Min Act: -0.1700

Data Index: 150273 (Open Web Text)

Max Activating Token Index: 241

Text #19

Max Range: 3.4587. Min Range: -3.4587

Max Act: 3.2626. Min Act: -0.1700

Data Index: 5066275 (Open Web Text)

Max Activating Token Index: 489