Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 682 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.1805. Min Act: -0.1493

Data Index: 1316037 (Open Web Text)

Max Activating Token Index: 259

Text #1

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.1318. Min Act: -0.1654

Data Index: 2808058 (Open Web Text)

Max Activating Token Index: 793

Text #2

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.1364. Min Act: -0.1682

Data Index: 6279644 (Open Web Text)

Max Activating Token Index: 232

Text #3

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.1243. Min Act: -0.1449

Data Index: 6828939 (Open Web Text)

Max Activating Token Index: 297

Text #4

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.1174. Min Act: -0.1555

Data Index: 1748913 (Open Web Text)

Max Activating Token Index: 197

Text #5

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.1056. Min Act: -0.1463

Data Index: 7150634 (Open Web Text)

Max Activating Token Index: 696

Text #6

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.1082. Min Act: -0.1682

Data Index: 4245929 (Open Web Text)

Max Activating Token Index: 260

Text #7

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.0999. Min Act: -0.1700

Data Index: 168337 (Open Web Text)

Max Activating Token Index: 247

Text #8

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.0976. Min Act: -0.1664

Data Index: 2597772 (Open Web Text)

Max Activating Token Index: 182

Text #9

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.1057. Min Act: -0.1456

Data Index: 3430943 (Open Web Text)

Max Activating Token Index: 750

Text #10

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.0954. Min Act: -0.1687

Data Index: 2685600 (Open Web Text)

Max Activating Token Index: 224

Text #11

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.1024. Min Act: -0.1592

Data Index: 3732599 (Open Web Text)

Max Activating Token Index: 225

Text #12

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.0927. Min Act: -0.1410

Data Index: 6120469 (Open Web Text)

Max Activating Token Index: 810

Text #13

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.0952. Min Act: -0.1634

Data Index: 4293672 (Open Web Text)

Max Activating Token Index: 914

Text #14

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.0895. Min Act: -0.1461

Data Index: 8358024 (Open Web Text)

Max Activating Token Index: 213

Text #15

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.0822. Min Act: -0.1579

Data Index: 457689 (Open Web Text)

Max Activating Token Index: 183

Text #16

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.0855. Min Act: -0.1673

Data Index: 8785303 (Open Web Text)

Max Activating Token Index: 232

Text #17

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.0827. Min Act: -0.1645

Data Index: 1969521 (Open Web Text)

Max Activating Token Index: 270

Text #18

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.0882. Min Act: -0.1586

Data Index: 7261408 (Open Web Text)

Max Activating Token Index: 222

Text #19

Max Range: 1.1805. Min Range: -1.1805

Max Act: 1.0873. Min Act: -0.1655

Data Index: 868337 (Open Web Text)

Max Activating Token Index: 281