Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2830 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.4203. Min Act: -0.1700

Data Index: 3652298 (Open Web Text)

Max Activating Token Index: 489

Text #1

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.3808. Min Act: -0.1700

Data Index: 6493806 (Open Web Text)

Max Activating Token Index: 567

Text #2

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.3536. Min Act: -0.1700

Data Index: 3251611 (Open Web Text)

Max Activating Token Index: 781

Text #3

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.2876. Min Act: -0.1700

Data Index: 7293413 (Open Web Text)

Max Activating Token Index: 128

Text #4

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.3002. Min Act: -0.1700

Data Index: 4107109 (Open Web Text)

Max Activating Token Index: 998

Text #5

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.2526. Min Act: -0.1700

Data Index: 488935 (Open Web Text)

Max Activating Token Index: 950

Text #6

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.2885. Min Act: -0.1700

Data Index: 3281989 (Open Web Text)

Max Activating Token Index: 60

Text #7

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.2518. Min Act: -0.1700

Data Index: 3272537 (Open Web Text)

Max Activating Token Index: 915

Text #8

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.2082. Min Act: -0.1700

Data Index: 3621618 (Open Web Text)

Max Activating Token Index: 273

Text #9

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.2234. Min Act: -0.1700

Data Index: 2199183 (Open Web Text)

Max Activating Token Index: 174

Text #10

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.1924. Min Act: -0.1700

Data Index: 1990621 (Open Web Text)

Max Activating Token Index: 261

Text #11

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.2007. Min Act: -0.1700

Data Index: 1878948 (Open Web Text)

Max Activating Token Index: 49

Text #12

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.2113. Min Act: -0.1700

Data Index: 1017030 (Open Web Text)

Max Activating Token Index: 80

Text #13

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.2184. Min Act: -0.1700

Data Index: 5543769 (Open Web Text)

Max Activating Token Index: 598

Text #14

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.1940. Min Act: -0.1700

Data Index: 8187494 (Open Web Text)

Max Activating Token Index: 893

Text #15

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.1778. Min Act: -0.1700

Data Index: 1615983 (Open Web Text)

Max Activating Token Index: 293

Text #16

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.1575. Min Act: -0.1700

Data Index: 3263475 (Open Web Text)

Max Activating Token Index: 537

Text #17

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.1445. Min Act: -0.1700

Data Index: 6698977 (Open Web Text)

Max Activating Token Index: 539

Text #18

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.1144. Min Act: -0.1700

Data Index: 6819966 (Open Web Text)

Max Activating Token Index: 503

Text #19

Max Range: 4.4203. Min Range: -4.4203

Max Act: 4.1549. Min Act: -0.1700

Data Index: 2022989 (Open Web Text)

Max Activating Token Index: 297